Reconciling conflicting worker results is the new reasoning step

Reconciliation is the step that appears the moment research is farmed out to several model calls running side by side. Each call reads a different body of material. None of them sees what the others found. The parent that collects their outputs can therefore be holding two accounts of one fact that were never in contact, neither of which is marked as disagreeing with anything, because neither author knew there was anything to disagree with. Deciding what the answer then says is a judgement about evidence rather than a continuation of a single line of reasoning, and it is a step somebody has to design.

The system this chapter works from is Scout, an internal competitive-research assistant at a company that sells analytics software to enterprises. A sales director asked Scout how a competitor called Vantage had changed enterprise pricing over the previous eighteen months. Scout dispatched five workers, and two of them touched pricing: one read archived copies of Vantage's public pricing pages, and one read sales-call notes in the company's own CRM, the customer-relationship management system where sales representatives log what happened on a call. Those two workers came back agreeing about the new price and disagreeing about when it took effect and about whom it applied to. Only one of the two accounts reached the answer, and nothing recorded that a choice had been made.

The arrangement, and why anyone builds it#

The arrangement that produced those two accounts is called orchestrator–worker. One model call, the orchestrator, reads the goal, decides at runtime what smaller pieces of research it implies, and emits them as subtasks. Each subtask goes to a separate model call, a worker, which runs with its own tools in its own fresh context, knows nothing about its siblings, and returns findings rather than a finished answer. A synthesis step then folds every worker's findings into the single answer the user reads. Orchestrator and workers are usually the same underlying model given different jobs, and the orchestrator is also called the lead or the parent agent.

Systems are built this way because breadth is expensive to carry in one context. A single agent asked about eighteen months of a competitor's pricing reads the archived pages, the press releases, the review sites and the CRM notes one after another, keeping every irrelevant paragraph it passed alongside the paragraph that mattered. Five workers read those five bodies of material at the same time, each discarding its own dross. You meet the arrangement in products that advertise deep research over the web, in competitive-intelligence assistants like Scout, and in coding agents that fan several searches across a repository before proposing an edit.

The property that creates the reconciliation problem is the same property that makes the arrangement cheap, and it is worth walking rather than asserting. A worker runs in a fresh context, so everything it read is held in that context and nowhere else. When the worker returns, its context is discarded and only its findings travel upward. The parent therefore receives a claim stripped of the material that produced it: it cannot re-read the page, cannot see what the worker rejected, and cannot tell how hard the worker had to reach. Because no worker could see its siblings, no returned finding carries a note saying another finding contradicts it. Put those two together and the consequence follows. A contradiction between workers is invisible in the returned documents, so it can only be found and settled by something the parent holds itself — and if the parent holds no rule, the ordering and phrasing of the concatenated outputs settle it instead.

Two accounts of one fact, side by side#

Here are the two records Scout's pricing workers returned, printed from the run as the synthesis step received them.

[
  {
    "subtask_id": "st_3", "scope": "archived pricing pages, vantage.com",
    "claim": "Enterprise floor moved from $48k to $60k/yr.",
    "effective_date": "2025-03-01", "confidence": 0.81,
    "evidence": "capture 2025-03-04: 'Enterprise — from $60,000/yr'"
  },
  {
    "subtask_id": "st_5", "scope": "CRM call notes, competitive-loss field",
    "claim": "Enterprise floor moved to $60k/yr for new logos only; renewals unchanged until anniversary.",
    "effective_date": "2025-01-22", "confidence": 0.66,
    "evidence": "opp 118204 note: 'their AE confirmed 60k floor, new business only'"
  }
]

Five fields are doing the work, and each is worth naming where it is used. subtask_id says which dispatched piece of work the record came from, st_3 being the archived-pages worker and st_5 the CRM worker. scope is the worker's own description of the material it was told to read. claim is the finding written as a sentence. effective_date is the worker's belief about when the change happened, which is a different question from when the worker saw the evidence. confidence is the worker's own number for how sure it is, on a scale where higher means surer. evidence is the fragment the worker is resting on: for st_3 a capture of a public page, for st_5 a rep's note quoting a Vantage AE — an account executive, one of the competitor's own salespeople.

Read the two records against each other rather than one at a time. They agree on the value: both say the enterprise floor is now sixty thousand dollars a year. They disagree on effective_date by about six weeks. And st_5 carries something st_3 does not carry at all, a restriction on whom the new floor applies to — new logos only, with renewals staying at the old floor until their anniversary. That third difference is not a disagreement in the ordinary sense, because st_3 never made a claim about renewals. It is a claim present in one record and absent from the other, which is a shape a comparison of two values cannot express.

Nothing in Scout decided between these. The synthesis prompt said write a clear, well-sourced answer from the research below, and the model did what that instruction permits, which is to pick the version that fits most comfortably into a paragraph. Three things pushed the same way here, and it is worth noticing that all three are properties of the record rather than of the world. The higher confidence score belongs to st_3. The cleaner date, the first of a month, belongs to st_3. And st_3 appeared earlier in the concatenated context. The carve-out — the part the account teams actually needed — dropped out without leaving a trace anywhere in the run.

What the sequential design never has to do#

The design review that produced Scout had two proposals on the table. Design A was the planner–executor split from the previous topic: a planner emits a sequence of research steps, and an executor carries them out one at a time, each step seeing what the last one produced. Design B was the orchestrator–worker shape described above, with the parent deciding the subtasks at runtime, dispatching workers concurrently, and synthesising what they return.

The argument in the room was about concurrency. Someone said that B needs correlation identifiers so results can be matched back to the subtasks that requested them, and deadlines so a worker that never returns does not hang the run, and that A needs neither. That is a comfortable thing to say and it is wrong in both halves. A sequential executor issues a step and waits for it, which means it also needs to know which step a returning result belongs to and how long it is willing to wait. An executor with a hung tool call is exactly as stuck as an orchestrator with a hung worker, and it will sit there just as long. Correlation and deadlines are what dispatching-and-awaiting implies, and both designs dispatch and await. They are the plumbing that concurrency makes more visible, not a capability one design has and the other does not.

The requirement B has and A does not is the one that produced the Vantage answer. A sequential executor never holds two independently produced accounts of the same fact at once. Each step in A is handed the state left by the step before it. So when step four contradicts step two, it contradicts something already in its own context, and reconciling the two is an ordinary continuation of a single line of reasoning — the same reasoning that produced the earlier claim, now revising it.

An orchestrator's workers do not see each other. They produce their outputs in isolation, and the parent receives a set of accounts that were never in contact, some of which disagree, none of which is annotated as disagreeing because no worker knew there was anything to disagree with. Folding those into one answer is a judgement about evidence: which source is closer to the fact, which is more recent, which is more likely to be a paraphrase of the other, and whether the disagreement is a contradiction at all or two true statements about different populations. That last question is the one the Vantage records actually turned on. That judgement has to be made by something, and in Scout it was being made by whichever ordering and phrasing the concatenation happened to produce.

The diamond in B is the whole of the difference. In A there is no node like it, because the merging happened incrementally inside the sequence and was never a separate thing anyone had to design. Everything in A's column is a chain, and a chain has no place where two independently written claims meet for the first time.

The map–reduce analogy breaks at the reducer#

The comparison that misleads people here is map–reduce, and it is worth handling as an analogy rather than as a description, because the resemblance is real and stops at exactly one place. Map–reduce is the batch-processing shape in which the same operation runs over many pieces of data in parallel — the map — and a second function combines the results — the reduce. It misleads because the shapes are identical. Work fans out, results come back, something combines them.

The break is at the reducer. In map–reduce the reducer is a total function chosen in advance — sum, concatenate, take the maximum — and it is defined for every pair of inputs including contradictory ones, because it does not have a notion of contradiction. Summing 48,000 and 60,000 is well-defined and meaningless, and that is the point: the reducer's correctness is somebody else's problem, guaranteed by the mapper producing values that are safe to combine that way.

Worker outputs are not safe to combine that way. They are natural-language claims with implicit scope, and combining them requires deciding what each one actually asserts before you can tell whether two of them conflict. Consider a smaller case from the same run. One worker reported that Vantage "removed the data-residency add-on"; another reported that Vantage "now includes data residency in Enterprise". Those look contradictory in a list of bullet points and are the same fact stated from two vantage points, and telling that apart is a semantic operation that a reducer signature has no room to express.

The same asymmetry shows up in what a failure means. If a mapper in a batch job dies, the framework re-runs it and the reduce is unaffected, because the reducer is a pure function of whatever set it is eventually given and has no expectations about that set beyond its type. A synthesis step does have expectations, in the sense that its output is only meaningful relative to a set of subtasks somebody intended to cover the question. Scout's synthesis could not tell the difference between four workers and five, or between two workers agreeing and one worker's claim being echoed by another worker who had read the same press release. Those distinctions require information about the dispatch that never appears in the returned documents themselves, which is the thread the later chapters follow.

A window large enough to hold the contradiction#

There is also a temptation to reach for a larger context window as the answer to disagreement, on the theory that a model shown everything at once will notice the contradiction and handle it. A context window is the amount of text one model call can hold at a time, measured in tokens, the word-fragments a piece of text is chopped into before a model reads it.

That belief is worth taking apart rather than waving away, because it is the commonest wrong answer in this area and it fails a test the run itself already ran. Scout's five worker outputs came to roughly 14,000 tokens, comfortably inside the window, and the model saw both pricing claims in the same prompt. It did not notice, or rather it noticed and resolved the conflict by a policy nobody had written down and nobody could inspect afterwards. Capacity to see the inputs is not the same as a rule for what to do when they disagree.

The shape of that mistake is worth carrying to the rest of this topic. A quantity was held fixed and comfortable — the window, which fits the inputs several times over — while the property that actually decides the outcome varied freely, that property being whether any rule for resolving a disagreement exists. Anything that reads as a defence here is a defence against truncation, and truncation was never what went wrong.

Read that table by its gaps rather than by its cells. The first two rows have no gap at all: both columns say needed, and those two rows are the entire content of the argument the review made and got wrong. Every row from the third down does separate the columns, and they separate in one direction, because each is a consequence of the same fact — that B holds accounts which were produced without contact.

The last row is the one that decides how much of this topic is about engineering rather than prompting. Synthesis is a component. It has inputs you can enumerate, a policy it applies, and outputs you can inspect, and it is capable of being wrong in ways that none of the workers were wrong. Scout's workers both did their jobs correctly. The defect was entirely downstream of them, in a step that nobody had specified beyond a sentence in a prompt, and no amount of improving the workers would have touched it. What that row licenses is a change of engineering posture: a component with a name can be given a contract, a test and an owner. What it does not license is any confidence about the set arriving at that component, which the table says nothing about at all.

Anthropic's published account of building a multi-agent research system, How we built our multi-agent research system, is worth reading alongside this, because it describes the same asymmetry from the other side: the orchestrator's hardest work is not deciding what to farm out but deciding what the returned material collectively means, and that is where their engineering effort concentrated.

Fan-outs that have nothing to reconcile#

Not every fan-out has a reconciliation problem. If the subtasks partition a set of objects rather than a set of questions — summarise each of these forty support tickets, extract the parties from each of these twelve contracts — then two workers can never assert incompatible things about the same fact, because no fact is in two workers' scopes. The parent's job in that case really is a reduction: order the results, format them, hand them back. That shape is a fan-out with a deterministic combine step, and calling it an orchestrator–worker design overstates what it needs. The pattern in this topic earns its name when subtasks are questions about a shared subject, because that is the condition under which two workers can look at different evidence and reach different conclusions about the same thing.

It also does not apply when there is only ever one worker per run in practice, which is more common than it sounds in systems where the decomposition step usually returns a single subtask. A synthesis step over a set of size one is a paraphrase, and a paraphrase costs a model call and a chance to lose detail. If the traces show a median subtask count of one or two, the reconciliation machinery is overhead sitting on top of what is effectively a sequential run.

And reconciliation, when it does apply, does not solve coverage. Deciding well between two accounts of Vantage's pricing floor says nothing about whether anyone was assigned to look at Vantage's channel-partner discounts, which was the second thing the sales director needed and which no worker had been given. A synthesis step is only ever as good as the set it is synthesising, and it has no way to want something that is not in front of it.

That dependency runs backwards through the design. The quality of the reconciliation is bounded above by the decomposition, and not only through omissions. When two workers are handed scopes that overlap, the parent manufactures its own disagreements, because the same fact gets researched twice from different starting points and comes back in two forms that then have to be reconciled at cost. Scout's two pricing claims were partly an artefact of exactly that — st_3 and st_5 had both been told to establish current enterprise pricing, from different sources, with no statement of who owned the question.

Before the synthesis step can be built well, then, the set it receives has to be one somebody checked. That means treating the decomposition itself as an artefact with properties you can test, which is a different kind of object from a prompt and admits a different kind of check.