Pipeline, barrier, loop-until-dry: orchestration shapes
Fan-out comes in three shapes — barrier, pipeline and loop-until-dry. What each costs in wall clock, when a barrier earns it, and how to measure your own.
Four agents, and it finished in the time of the slowest one
When you divide work between four agents, each one finding a bug and each finding then getting verified, the slowest find and the slowest verification define the duration of the entire run even if it wasn't the same item that was slowest at finding and at verifying. Half of the agents spend most of the time waiting for another agent they didn't depend on.
The reason for this is we weren't explicit about the fact that the work should be run in parallel, and then verified — what we said was a single instruction which naturally translates into two steps synchronised by a barrier. It's the barrier which took the time despite not being part of the prompt.
The three shapes
There are three things you can say to split the work between agents, two of them happen by accident, and one only if you really choose it:
barrier
pipeline
loop-until-dry
The barrier is what happens by default. It's a natural thing to say as soon as we think of splitting sth between multiple agents — "run in parallel", "run concurrently", "run in parallel and then verify" etc, the barrier is just a wall before which all items need to wait until all are there before moving forward:
all items do stage 1
fan out
await all
fan out again
total time is sum of per-stage maximums
Everything else is the pipeline. It's a thing you need to write a little bit of code for to enforce, and it looks like this:
every item goes through every stage on its own
item A can be in stage 3 while item B is in stage 1
total time is the longest single chain; not sum of per-stage worst cases
The loop-until-dry is a thing that naturally happens if you don't know how many items there are, so you run rounds and stop when rounds stop finding anything new — a convergence condition rather than a fixed round count:
item count unknown
run rounds
stop when rounds stop yielding any new items
Anthropic has a catalogue of patterns where they list the barrier under parallelization, together with two sub-variants that are worth differentiating:
sectioning — splitting a task into independent subtasks to be done in parallel
voting — running the same task multiple times to collect various results
Sectioning is about dividing the work, and voting about dividing the judgment. Both are barriers, both can make sense, but they differ in the way they fail — sectioning that doesn't work well leaves gaps, and voting that doesn't work well gives false confidence.
What the barrier actually costs
The barrier is also the most expensive of the three. Let's measure it:
4 items
2 stages
sleep to represent agents so the only variable is the shape
stage times picked so the worst item in stage 1 isn't the worst in stage 2 — a regular setup, not a rigged one
The pipeline takes 10s (wall clock) with 4 workers, the busiest item spends 10s there and none of it waiting:
item stages (s) busy idle
a 8.0 1.0 9.0 0.0
b 3.0 7.0 10.0 0.0
c 2.0 2.0 4.0 0.0
d 2.0 1.0 3.0 0.0
pipeline: 10.0s wall clock, 4 workers, busiest item 10.0s, 0.0s of that spent waiting.The barrier with the same stages and number of workers takes 15s, and the busiest item spends there 10s, but 5s of it idly:
item stages (s) busy idle
a 8.0 1.0 9.0 6.0
b 3.0 7.0 10.0 5.0
c 2.0 2.0 4.0 11.0
d 2.0 1.0 3.0 12.0
barrier: 15.0s wall clock, 4 workers, busiest item 10.0s, 5.0s of that spent waiting.10s vs 15s for the same work, and the idle column is more honest as it's actually 0s for the pipeline and 34s (6 + 5 + 11 + 12) of billed waiting for the barrier. If we were to assume there are enough workers so nothing queues, then the barrier is 8 + 7 (worst item per stage), and the pipeline is 3 + 7 (longest chain of one item). They are equal only if a single item is the worst at every stage; whenever different items are worst at different stages — which is what fanning out over different files looks like — the difference between the two is pure loss.
The more items you have, the higher the penalty is. Adding another item can only increase the maximum time for each stage, but the longest chain of a single item can get longer only if the new item is slower than all existing ones to do the entire thing.
When a barrier is the right answer
These are all valid use cases for the barrier:
a stage is a barrier if it needs sth from items apart from the one it's currently operating on
dedupe before a costly stage if 5 people found the same bug in 3 places so verifying it three times triples the cost and doesn't give any more information but the merge really requires results from all of them
early exit on zero if you need to wait for every agent to report there's nothing before proceeding
prompt comparing items — think sorting findings by importance, or selecting a root cause from among them
human sign-off between stages, as Claude Code's workflow runtime says: "For sign-off between stages, run each stage as its own workflow"
There are also things that resemble the above but aren't:
"we need to flatten and filter the results first" — a transformation, not a dependency, do it within the next stage
"the stages are conceptually different" — pipelines are built from separate stages, different doesn't mean synchronised
"it's more readable this way" — the barrier's latency is real, and the readability benefit is for the author, not the reader
If you think about it, between two fan-outs, if it's a map, filter or flatten with no cross-item logic, the barrier was probably unnecessary. If it's dedupe, sorting based on comparing items, or counting and branching — it was needed.
Loop-until-dry
The loop-until-dry is the most natural thing if you know there's an unknown number of items to work on. If you ask to "find bugs", there's no bug count, and both of these seem like sensible ways to stop:
stop at ten findings — which is you asserting there are ten
stop after three rounds — which is you asserting the tail is empty because you stopped looking
Neither of them works. The only thing that makes sense is to define a condition and keep running rounds until some number (let's say two) of consecutive rounds didn't find anything new:
unknown item count
run rounds
stop when X consecutive rounds didn't yield any new items
And Claude Code's documentation frames a task about flaky tests identically to it:
"and stop once two rounds in a row find nothing new"
If we were to think of the loop-until-dry as a regular loop, and visualise it running for four rounds, it'd look like this:
round 1: 2 new, 2 total, 0 dry
round 2: 1 new, 3 total, 0 dry
round 3: 0 new, 3 total, 1 dry
round 4: 0 new, 3 total, 2 dryAnd you can see it's actually the second round that makes the case for this shape. A fixed count of one round would have missed a third of the findings; the loop got them only because it was still asking.
What you need with the loop-until-dry is one cumulative set of keys, which you add to before judging anything and compare everything against. You must not dedupe against the survivors, as then every finding you rejected in the previous round becomes fresh again next round, gets rejected again, etc, until the loop runs forever waiting for sth else to happen.
And you need to put a hard cap above the convergence condition. The loop is a wager that the finder will run out of ideas eventually; you can set a cap on rounds, or agents, or on the budget so the condition has a chance to make its regular decision. That's what runtimes do — Claude Code's workflows have a run limit of 1,000 agents, and the reason the docs give for the number is that it "prevents runaway loops".
Being wide is not free
Every shape from this post multiplies token count too, not just the duration. Anthropic's figures show 4× for an agent and 15× for a multi-agent system (compared to a chat) — but these are the numbers before we know if the second agent has contributed anything.
We measured the simplest possible agent, it was a prompt telling it to reply with a single word, touching nothing:
Claude Code 2.1.220
tools:disabled
4566 cached input tokens for 5 output ones
$0.002986
Codex CLI 0.146.0
same prompt
sandbox:read-only
4050 tokens used
That's the cheapest conceivable agent — it takes a little over four thousand tokens to start an agent and get a word from it, and then you multiply it by how wide the fan-out is before you can do anything useful.
This aligns with what other vendors say:
OpenAI: because each subagent does its own model work and tools work, subagent workflows consume more tokens than single-agent runs that would do the same
GitHub: in fleet mode, dividing work may result in more LLM interactions than if the main agent did it
Moonshot: simple tasks don't need a dispatched subagent, the main agent does them more efficiently
Anthropic's own failure mode is worth keeping in view too — they report agents:
"spawning 50 subagents for simple queries"
If you leave the width of a fan-out to the agent, the agent will pick it for you.
The shape doesn't fix a bad split
The shape can't fix a bad decomposition. There's a serious argument that you shouldn't be doing any of this, and it's worth meeting rather than skipping — Cognition's Walden Yan published it as two principles:
"Share context, and share full agent traces, not just individual messages"
"Actions carry implicit decisions, and conflicting decisions carry bad results"
His example is a Flappy Bird clone split between two subagents, one of which builds a Super Mario Bros background while the other builds a bird that doesn't look like a game asset, and then the final agent has to reconcile outputs that were never going to fit:
"subagent 1 and subagent 2 cannot see what the other was doing and so their work ends up being inconsistent"
This is spot-on, except it's orthogonal to shape — barrier, pipeline and loop-until-dry determine when agents run, but don't determine what they know, and no scheduling can make two uninformed agents agree.
What this comes down to is what you split on. If you split on the axis where two branches need to yield things that are compatible with each other, you must either share the context between them, or they're not two branches; Anthropic put it from the opposite perspective, saying:
"most coding tasks involve fewer truly parallelizable tasks than research"
If you split on this axis, a pipeline is safer than a barrier, not just faster — every item brings its own context to every stage, so no stage can blend items that aren't connected and force a later agent to guess which decision came from where.
Measuring your own
To run your own experiments, here's a script:
short enough to read in one sitting
feeds an array of items through stages in any shape you call it with
interpolating
{item}into every stage command, andpiping each stage's stdout into the stdin of the next stage
You can begin with sleeps as they don't cost anything, so it's the shape that's under the test; the times per item are the entire exercise, if you give every item an equal time both shapes will take the same (the only case when a barrier is free):
python3 scripts/shapes.py --shape pipeline --workers 4 \
--stage 'case {item} in a) sleep 8;; b) sleep 3;; *) sleep 2;; esac; echo found-{item}' \
--stage 'case {item} in b) sleep 7;; c) sleep 2;; *) sleep 1;; esac; echo checked-{item}' \
-- a b c dAnd here's the same run, with sleeps replaced by the harness's headless command that audits every file and then tries to refute the finding so it's some actual work; in this case it's the later stage reading the earlier stage's stdout from stdin:
python3 scripts/shapes.py --shape barrier --workers 4 \
--stage 'claude -p "Audit {item} for missing auth checks" --output-format json' \
--stage 'claude -p "Try to refute this finding: $(cat)" --output-format json' \
-- src/routes/a.ts src/routes/b.ts src/routes/c.tsRun both shapes over the same work before picking one. The difference between the totals is what the barrier costs, and it's usually higher than you think.
The shapes in Claude Code
The Claude Code is all about parallelisation, and it comes in two flavours: subagents within a session, and dynamic workflows — where the actual pipelining happens.
Subagents within a session are of a barrier type by default since version 2.1.198 (they run in the background unless Claude needs the output to proceed, which is the barrier case and also the most common one, because subagent results land in the chat anyway), and they have a few limits as of 2.1.220: up to 20 can be run at once, up to 200 per session and you can nest them three levels deep under the main conversation (so you can have a review subagent which runs a verifier subagent for every finding for example, keeping all of its intermediate output out of your context and returning just a summary).
Dynamic workflows are where the actual pipelining happens. The Claude Code is based on two primitive script commands to run subagents: agent() for running a single subagent and pipeline() for running a subagent per list item. These are very powerful, but what's important is that thanks to them, all loops, conditionals and intermediate results can be in the script — as the docs put it, "so Claude's context holds only the final answer". In terms of limitations, it's up to 16 (fewer if you run it on a machine with limited CPU cores) agents at once, and 1,000 per run; the default size guideline aims at under 15.
When it comes to orchestrating this at the script level, the building blocks of the CLI are:
running prompts headlessly with results as JSON (
claude -p --output-format json);using the
--max-budget-usdflag whenever you want to run something fan-out-ish without supervision;claude --bgto run a session in the background;listing all active sessions as JSON with
claude agents --json— "for scripting; does not require a TTY".
Gotcha: stopping a workflow in the middle of its fan-out phase is more expensive than you might think. The thing is that replay follows the order agents were started, so cached results stop at the first agent that didn't finish, and "every agent that started after that one runs again, even if it completed".
That's why the docs conclude that a run split across many small agents "preserves more progress than one long agent" — so breadth delivers not only speed but resumability.
The shapes in Codex CLI
A custom agent is just a TOML file placed in a certain location:
~/.codex/agents/for personal usage.codex/agents/for project-level usage
The TOML file must contain 3 mandatory keys: name, description and developer_instructions. You can define a custom agent and then parallelise its usage by saying sth like "spawn one agent per point" or by using the AGENTS.md file to define the directive.
The behaviour of such parallelisation is defined as a barrier in the documentation. It means that for example if you have 3 sub-agents, they will be run in parallel but their output will be merged into a single response and the main thread will wait until all sub-agents have finished. This is a reasonable default behaviour when comparing things but not if you need to perform independent per-file work.
There's also another property of custom agents — agents.max_concurrent_threads_per_session which limits the number of threads that can be spawned simultaneously (excluding the main thread) during a single session. If this property isn't defined, Codex will use its default value.
If you want a pipeline instead, you script it yourself. The main building block of such a script is the codex exec command:
codex exec --json— prints an event stream in JSONL formatcodex exec --output-schema <FILE>— limits the final message to a certain schema so it can be parsedcodex exec -o / --output-last-message <FILE>— outputs only the last message of the runcodex exec --ephemeral— doesn't create a session for the run so a hundred disposable runs stay out of the session listcodex exec -C <DIR>— runs the pipeline in a separate directory for each run
Gotcha, and we walked into it: if you use codex exec and don't provide a prompt as an argument, it will take its instructions from stdin — and it reads a piped one either way. This works well if you pipe some input to the command on purpose, but not if you do it by accident. Ours announced "Reading additional input from stdin..." before the run started, on a pipe we hadn't meant to give it. To avoid it, just redirect < /dev/null in every stage of your pipeline that isn't deliberately being fed.
The shapes in GitHub Copilot CLI
1.0.77 of GitHub Copilot CLI contains an "Agents / Subagents" section in its interactive command list, and /fleet is the key to it — it "enables fleet mode for parallel subagent execution"; /tasks watches them and /subagents sets which model each one gets.
Given an implementation plan, GitHub describes it as breaking that plan "down into smaller, independent tasks that can be executed in parallel by subagents". It's the main agent that assesses the subtasks and their dependencies, and runs in parallel whatever can be.
The orchestrator is responsible for splitting the plan, so there's no control from the user side in terms of how wide the fan-out gets.
But if you need to set up pipelines, you can achieve it by creating a script using non-interactive mode of the CLI: copilot -p, and setting output format to json (--output-format json) to get a JSONL which is easy to process. Then you can use -s to get only responses and --max-ai-credits to limit how much it can spend during a single run.
There are also 2 pitfalls around the feature:
GitHub itself says in their docs that "if your request is inherently sequential, using the /fleet slash command mode may not provide any benefit" — and every sub-agent talks to the LLM separately, so splitting work up "may result in more LLM interactions than if the work was handled by the main agent". A decomposition that didn't need to happen still bills as though it did.
The help text says that for non-interactive runs you need
--allow-all-tools, so if you set up a script that does fan-out, by default it's an ungated fan-out across every agent you started — narrow it with--allow-tooland--deny-toolbefore you widen the fan.
The shapes in Cursor
Sub-agents are stored in project-level and user-level directories:
Project level:
.cursor/agents/User level:
~/.cursor/agents/For compatibility, Cursor also reads
.claude/agents/and.codex/agents/If there's a conflict, the project level definition wins
It's good to check this before you get into debugging problems with running multiple agents in parallel. Especially if your repo is set up for another tool, it might provide agents that aren't defined locally.
In terms of implementation: every sub-agent "operates in its own context window, handles specific types of work, and returns its result to the parent agent". You can run multiple sub-agents simultaneously — the docs pitch it as working "on different parts of your codebase without waiting for sequential completion". A sub-agent can spawn its own sub-agents but those grandchildren aren't allowed to spawn any further ones. Make sure to check the docs before you plan a three-level fan-out based on it.
The headless agent run with JSON output (agent -p --output-format json) is the most atomic element of scripting. Two more things are worth having in your fan-out: the create-chat command, which creates an empty chat and returns its ID so you can have one per thing you work on, and the -w/--worktree flag, which tells Cursor to create a worktree in ~/.cursor/worktrees/<reponame>/<name> and run the agent there. That one is your protection against multiple agents altering the same files.
Gotcha: background sub-agents "return immediately" but still continue their work. It's a different contract from the foreground ones. So if you think they returned, it doesn't mean that they finished, and you might end up with some stages of your pipeline being run concurrently by accident.
The shapes in Gemini CLI
Subagents are Markdown files with YAML frontmatter either in ~/.gemini/agents (per-user) or in .gemini/agents (per-repo). You can reach them via the primary agent:
automatically — you let it route to whichever ones fit;
manually — using the
@agentsyntax, for example "@frontend-specialist Can you review our app and flag potential improvements?".
The latter way is useful when you want to pin a fan-out to the subagents you meant instead of the ones the model would pick.
Google's line is that the CLI "supports parallel subagents, allowing you to spin off multiple subagents or many instances of the same subagent, at the same time."
If you need to run a subagent non-interactively (as part of an automated workflow for example), there are a few shapes you can achieve using the scripting layer:
single prompt invocation with
gemini -p— this is the atomic unit;outputting either machine-readable JSON with
-o jsonor streaming JSON with-o stream-json;running everything in a worktree using the
-w/--worktreeflag (which is especially useful in the context of subagents).
Gotcha: it's Google's own, and it's about writes. "Exercise caution with parallel subagents for tasks that require heavy code edits. Multiple agents editing code at the same time can lead to conflicts and agents overwriting one another. Parallel subagents will also lead to usage limits being hit faster as requests are being sent in parallel across agents."
So we treat the width of a fan-out as a rate-limit decision, no different from the spend limit.
Antigravity CLI has similar constructs, but with different names:
subagent: truein the config — this is how you define an agent that can be called by the primary agent using itsinvoke_subagenttool;Tasks — the background things that are not agents, but "direct shell commands, testing suites, or simple background queries";
the location of these definitions is slightly different too: per-workspace they live in
.agents/agents/<name>.mdfiles, globally they're in~/.gemini/config/agents/.
Given the differences above, it's important to remember that the definitions made for one tool are not visible in the other.
The shapes in Kimi Code CLI
The sub-agent architecture in Kimi Code CLI is actually really cool — there are three agents available out of the box:
coder(edits)explore(read-only exploration)plan(design work, no shell access)
And as the docs say, "multiple sub-agents can run in parallel without interfering with each other". Basically what happens is that a sub-agent receives a task description from the main agent, works in its own isolated context, and then returns its conclusions; its intermediate reasoning and tool call records never mix into the main agent's history.
This way if you have your own agents — they are resolved in a specific order:
--agent-file.kimi-code/agentsor.agents/agentsin the projectextra
$KIMI_CODE_HOME/agentsor~/.agents/agentsplugin
built-in
To script it the base command is kimi -p and the best output format for sub-agents is --output-format stream-json; you can also use --agent <name> or --agent-file <path> to choose an agent profile per run, so different stages of your pipeline can use different agents without editing the config.
Gotcha, and it's a useful one: during a fan-out, the sub-agents save their runtime state under the agents/ directory in the session folder — one directory per instance with a wire.jsonl file containing the prompts, message history and the final state. This is what you analyse if you're not satisfied with the fan-out's results (e.g. you see that 3 out of 5 agents have run the same search).
Duplicated work is the failure mode a fan-out hides best, and it only shows up in the per-agent record. Moonshot also states the counter-case plainly, which is worth holding onto before you fan out at all: "For simple tasks, there is no need to dispatch a sub-agent — the main Agent handles them more economically."
Pick the shape from the dependency graph
You can decide by looking at your dependency graph:
does this stage need sth from items apart from the one it handles?
if no — it's a pipeline (which covers most stages)
if yes — it's a barrier, at that single seam only
if you don't know how many items there are — it's the loop-until-dry with a cumulative seen set and a hard cap
The barrier is where you naturally land. Ask any of these six harnesses for several workers at once and, wherever it needs their answers to carry on, the turn stops until the last of them is in.
The pipeline is the one you mostly have to build. Claude Code's workflow runtime is the only one of the six that hands you a function actually called pipeline(); everywhere else it's a script you write.
scripts/shapes.py#!/usr/bin/env python3
"""Run a list of items through stages in one of three shapes, and time it.
# every item walks its own stages, nobody waits
python3 scripts/shapes.py --shape pipeline \
--stage 'sleep 6; echo {item}' --stage 'sleep 2; echo ok' -- a b c
# every item waits for the slowest before the next stage starts
python3 scripts/shapes.py --shape barrier ...same... -- a b c
# keep running rounds until two in a row find nothing new
python3 scripts/shapes.py --shape until-dry --dry-rounds 2 \
--stage 'your-finder-command' -- sweep
`{item}` is substituted into each stage command; the previous stage's stdout
arrives on the next stage's stdin. Swap the sleeps for `claude -p`, `codex exec`
or whatever you drive and the numbers become yours.
"""
import argparse
import subprocess
import sys
import time
from concurrent.futures import ThreadPoolExecutor, as_completed
def run_stage(command, item, stdin):
"""One stage for one item. Returns (stdout, seconds)."""
started = time.monotonic()
done = subprocess.run(
command.replace("{item}", item),
shell=True,
input=stdin,
capture_output=True,
text=True,
)
return done.stdout.strip(), time.monotonic() - started
def pipeline(stages, items, workers):
"""Each item walks every stage on its own. No stage waits for its peers."""
def walk(item):
carried, spent = "", []
for stage in stages:
carried, seconds = run_stage(stage, item, carried)
spent.append(seconds)
return item, carried, spent, 0.0
with ThreadPoolExecutor(max_workers=workers) as pool:
return [f.result() for f in as_completed(pool.submit(walk, i) for i in items)]
def barrier(stages, items, workers):
"""Every item finishes stage N before any starts stage N+1."""
carried = {item: "" for item in items}
spent = {item: [] for item in items}
idle = {item: 0.0 for item in items}
with ThreadPoolExecutor(max_workers=workers) as pool:
for stage in stages:
reached = time.monotonic()
futures = {pool.submit(run_stage, stage, i, carried[i]): i for i in items}
for future in as_completed(futures):
item = futures[future]
carried[item], seconds = future.result()
spent[item].append(seconds)
released = time.monotonic() - reached
for item in items:
idle[item] += released - spent[item][-1]
return [(i, carried[i], spent[i], idle[i]) for i in items]
def until_dry(stages, items, workers, dry_rounds):
"""Rounds until `dry_rounds` in a row turn up nothing new.
Dedup against everything ever seen, not against what survived the last
round -- otherwise a finding you rejected comes back every round and the
loop never converges.
"""
seen, results, dry, rounds = set(), [], 0, 0
while dry < dry_rounds:
rounds += 1
found = pipeline(stages, items, workers)
fresh = [line for _, out, _, _ in found for line in out.splitlines() if line not in seen]
seen.update(fresh)
results.extend((f"{i}#{rounds}", out, spent, idle) for i, out, spent, idle in found)
dry = dry + 1 if not fresh else 0
print(f"round {rounds}: {len(fresh)} new, {len(seen)} total, {dry} dry", file=sys.stderr)
return results
def main():
parser = argparse.ArgumentParser(
description=__doc__, formatter_class=argparse.RawDescriptionHelpFormatter
)
parser.add_argument("--shape", choices=["pipeline", "barrier", "until-dry"], default="pipeline")
parser.add_argument("--stage", action="append", required=True, help="shell command; repeatable, in order")
parser.add_argument("--workers", type=int, default=4, help="how many run at once")
parser.add_argument("--dry-rounds", type=int, default=2, help="until-dry: empty rounds before stopping")
parser.add_argument("items", nargs="+")
args = parser.parse_args()
started = time.monotonic()
if args.shape == "pipeline":
results = pipeline(args.stage, args.items, args.workers)
elif args.shape == "barrier":
results = barrier(args.stage, args.items, args.workers)
else:
results = until_dry(args.stage, args.items, args.workers, args.dry_rounds)
total = time.monotonic() - started
print(f"{'item':<16}{'stages (s)':>28}{'busy':>8}{'idle':>8}")
for item, _, spent, idle in sorted(results):
stages = " ".join(f"{s:.1f}" for s in spent)
print(f"{item:<16}{stages:>28}{sum(spent):>8.1f}{idle:>8.1f}")
busiest = max(sum(spent) for _, _, spent, _ in results)
print(f"\n{args.shape}: {total:.1f}s wall clock, {args.workers} workers, "
f"busiest item {busiest:.1f}s, {total - busiest:.1f}s of that spent waiting.")
if __name__ == "__main__":
main()