Skip to content

[Feature Request]: Add Scatter-Gather Design Pattern #3577

Description

Name of the Design Pattern

Scatter-Gather Pattern

Category

Concurrency / Integration Patterns

Intent

The Scatter-Gather design pattern is a messaging and integration pattern used to parallelize task execution. It broadcasts (scatters) a single request to multiple recipients or worker services concurrently, and then aggregates (gathers) their responses back into a single response. This significantly reduces overall response time when fetching data from multiple independent sources.

Detailed Explanation

In enterprise architectures, a client request often requires gathering information from multiple downstream services or running multiple independent calculations. Running these calls sequentially can lead to high latency.

The Scatter-Gather pattern addresses this by:

  1. Scatter: Spawning multiple parallel threads or asynchronous tasks (workers) to call each downstream service or execute task partitions.
  2. Gather: Waiting for all tasks to finish (or timing out if some are slow), collecting the results, and passing them to an aggregator.
  3. Aggregate: Merging individual responses into a unified result format returned to the client.

Real-World Example

Consider a travel booking aggregator (like Kayak or Expedia). When a user searches for a flight, the portal scatters the query to multiple airlines' API servers simultaneously. It gathers the prices, sorts them, and displays the complete list to the user.

Conceptual Architecture

Here is a conceptual design for the Scatter-Gather pattern:

classDiagram
    direction TB
    class TaskSupplier {
        <<interface>>
        +execute() String
    }
    class AirlineService {
        -String airlineName
        -double basePrice
        +execute() String
    }
    class ScatterGatherExecutor {
        -ExecutorService executor
        +scatterGather(List~TaskSupplier~ tasks, long timeoutMs) List~String~
    }
    class Aggregator {
        +aggregate(List~String~ results) String
    }
    
    TaskSupplier <|.. AirlineService
    ScatterGatherExecutor --> TaskSupplier : scatters tasks
    ScatterGatherExecutor .. Aggregator : passes results to
Loading

Metadata

Metadata

Assignees

No one assigned

    Projects

    Status
    Todo

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions