One worker's fabricated statistic becomes the answer's claim

Scout is an internal competitive-research assistant, built on the orchestrator–worker arrangement: one model call decides what smaller pieces of research a goal implies, each piece goes to a separate model call with its own tools and its own fresh context, and a synthesis step folds what comes back into one answer. Chapter 5 gave those workers a shape to return findings in — atomic claims carrying a subject, a predicate, a value and an evidence object — so that two findings about the same fact could be made to collide and be resolved by a rule. This chapter is about a claim that collides with nothing because there was nothing else in the world for it to collide with.

A hallucination is a claim a model states as fact with nothing behind it, produced by the same machinery that produces its true claims and wearing the same confident phrasing. The word is usually applied to a single model answering a question. In a fan-out the same event has a different shape and a different remedy, because the context that produced the claim is thrown away before anyone downstream sees the claim. Scout produced one of these and it reached a board deck: a 34% figure attributed to a customer survey that does not exist.

Loud failures and quiet ones are not the same engineering problem#

A timeout announces itself by not arriving. Malformed output announces itself at the parser. An empty result announces itself as a zero. Each of those is detectable by code that knows nothing about the domain, which is why chapter 4's outcome states could be a small enumeration handled uniformly.

Fabrication has none of that structure, and the reason is worth stating as a property rather than as a complaint. Every check a system runs on a worker's response is a function of the response. A schema check reads the fields. A parser reads the syntax. A deadline reads the arrival time. All three are computable from the bytes that came back. A fabricated claim differs from a true one in exactly one respect — its correspondence to something outside the system — and correspondence is not a function of the bytes. No property internal to the response distinguishes them, so no check that only reads the response can either.

Fluency actively works against detection. A model that produces a confident, specific, well-formatted claim produces the same shape whether the underlying evidence exists or not, and specificity reads as credibility to both the synthesiser and the reader. A precise figure with a named source and a quarter attached is more persuasive than a hedged true finding beside it, and the persuasion comes from the form rather than from the fact. The 34% figure was more persuasive than the true findings around it precisely because it was crisper.

The fan-out makes this worse than it is for a single agent, and the causal chain is short enough to check link by link. A worker runs in its own context, so the documents it read are in that context and nowhere else. The worker returns findings rather than its context, so the fabricating worker's context is discarded the moment it returns. The parent therefore inherits a claim with no access to what the worker actually saw. And the other workers were sent elsewhere on purpose, so none of them is holding material that bears on this claim either. Isolation is the reason the pattern parallelises, and it is the same property that strips a fabrication of everything that might have caught it. In a single-agent run the invented statistic at least sits in the same context as everything else the agent read, and there is a chance the surrounding material contradicts it.

Make the evidence span checkable, not just present#

The evidence field in the previous chapter's claim schema was already there, and it did not help, because nothing verified it. This is the stumble worth naming, and it is tempting for a reason. A field that is always populated, always well-typed, and always plausible looks like a control. It is not one. A model asked to emit an evidence object will emit an evidence object; producing a plausible URL and a plausible-sounding span costs it nothing and is exactly the kind of pattern-completion it is good at. The quantity that stays constant is the presence and the shape of the field. The quantity that varies is whether the field corresponds to anything, and nothing in the pipeline was reading that.

The field only becomes a control when the parent can check the span against a document the run actually retrieved. That requires one piece of plumbing: workers fetch through a shared retrieval layer that caches every document body for the life of the run, keyed by a document id the worker must cite.

@dataclass(frozen=True)
class Evidence:
    kind: Literal["primary_quote", "reported_speech", "inference"]
    doc_id: str          # returned by the retrieval layer, not chosen by the worker
    span: str            # must appear verbatim in doc_id's body
    retrieved_at: str

def normalise(text: str) -> str:
    return " ".join(text.replace("’", "'").replace("“", '"')
                        .replace("”", '"').split()).lower()

def check_span(claim: Claim, docs: DocCache) -> SpanCheck:
    doc = docs.get(claim.evidence.doc_id)
    if doc is None:
        return SpanCheck.UNKNOWN_DOC        # cited a doc no worker fetched
    if normalise(claim.evidence.span) in normalise(doc.body):
        return SpanCheck.VERBATIM
    return SpanCheck.NOT_FOUND              # doc real, span not in it

The comment on doc_id carries the whole design. A worker that chooses its own identifier can invent one as easily as it can invent a figure, and the check would then be comparing a fabrication against a fabrication. An identifier minted by the retrieval layer cannot be invented, because the only way to obtain one is to have fetched something. That is what turns UNKNOWN_DOC from a missing-record error into a finding: a claim citing a document no worker fetched is a claim whose source the run never saw.

The normalisation is doing unglamorous work and skipping it produces a check nobody trusts. Smart quotes, non-breaking spaces, soft hyphens and collapsed newlines account for most of the false alarms in a naive substring match, and a validator with a 20% false-positive rate gets switched off within a week. Note what normalise does and does not do: it folds whitespace, unifies quote characters and lowercases, and it does not touch word order or wording. The check stays a substring test on a canonical form, which is why it is deterministic and why it costs microseconds.

Uncontested is not the same as corroborated#

Span validation catches claims with no textual basis. It says nothing about a claim that quotes a real span from a source that is itself wrong, or a claim whose span is real and whose interpretation of it is not — quoting "we are reviewing our packaging" and asserting a price rise is a verbatim span supporting an invented conclusion. For those, the useful signal is how many independent workers reached the same place.

The second stumble lives here, and it hides inside the merge machinery of the previous chapter. A claim that nothing contradicts passes every rule in that table untouched, and passing looks like agreement. It is not agreement. A claim that stands alone is, by construction, the unanimous view of everything that spoke, and one voice is unanimous in exactly the same formal sense that a hundred are. The system records "no conflict" and a reader hears "corroborated", and those are different facts about different populations.

That does not make single-sourced claims suspect as a class. Most true findings are single-sourced, since a decomposition that is doing its job sends one worker to each question, and treating every uncontested claim as unproven would flag almost everything. The rule worth having is narrower, and it targets the claims where being wrong is expensive.

def is_load_bearing(claim: Claim, answer: Answer) -> bool:
    return (
        claim.claim_id in answer.cited_claim_ids
        and (claim.is_numeric or claim.attributes_to_named_entity)
        and claim.question_id in answer.required_question_ids
    )

def gate(claims: list[Claim], answer: Answer, budget: int) -> list[Claim]:
    by_fact = group_by(claims, key=lambda c: (c.subject, c.predicate))
    flagged = [c for c in claims
               if is_load_bearing(c, answer) and len(by_fact[(c.subject, c.predicate)]) == 1]
    return flagged[:budget]

is_load_bearing is three conditions joined by and, and each one narrows the population for a separate reason. The claim has to appear in the answer, because a claim nobody reads cannot mislead anybody. It has to be numeric or attributed to a named entity, because those are the assertions that get repeated verbatim elsewhere. And it has to belong to a question the plan marked required, because that is where the goal said the answer's weight lay. Drop any one of the three and the gate flags material it has no budget to check.

Flagged claims go to a verification worker with a single instruction: find independent primary evidence for this exact assertion, or report that none exists. The result determines the claim's treatment rather than being advisory. Corroborated claims are stated plainly; uncorroborated ones are either dropped or marked as single-sourced in the answer with the source named, which is a materially different sentence for a reader deciding what to repeat in a board meeting.

The budget cap is not a nicety. Without it, a run whose answer cites forty numeric claims dispatches forty verification workers and doubles the token bill, and the ordering by which claims get the budget should be explicit — Scout sorts by whether the claim is numeric and appears in the answer's first two paragraphs, on the reasoning that those are the assertions most likely to be forwarded.

Grounding is a relation between a claim and a document#

The alternative that gets proposed at this point is a grounding judge: a second model call that reads each claim with its cited document and rules on whether the document supports it. Scout built one, because the span check cannot handle claims that legitimately summarise a whole page rather than quoting a line. Measured against the same 61 known-bad claims plus a sample of 200 known-good ones, the judge caught 29 of the 44 outright fabrications and flagged 11 good claims as unsupported, at about 900 tokens per claim — roughly $2.20 per 100 claims and a 5.5% false-alarm rate.

Set those results beside the span check's and the comparison is one-sided on every column that can be compared. The deterministic check caught all 44 where the judge caught 29. It raised no false alarms where the judge raised 11. It costs nothing where the judge costs $2.20 per 100 claims, and it runs in microseconds where the judge runs a model call. A component beaten on every measured column is not a component to build first, and a team that builds the judge first usually never builds the check, having spent its fabrication budget on the more interesting component.

The judge is still worth having, and the reason is the one thing the columns do not compare: population. A claim that summarises a page has no verbatim span to find, so the span check returns NOT_FOUND on a claim that is perfectly well grounded, and no amount of normalisation repairs that. The judge is the second layer over claims that pass the span check and make broad summarising assertions. That is a defensible reason to build a dominated component, and it is the only one.

Fabrications concentrate rather than spreading evenly, which gives a cheap prior. Of the 44, 31 came from workers whose sources_reached was two or fewer — workers that found little and, having been asked a question and given a schema to fill, filled it. That pattern is a direct consequence of how the worker prompt is written: an instruction to report findings creates pressure to have findings. Adding an explicit, first-class way for a worker to return nothing, with state: "empty" from chapter 4 as a legitimate and non-penalised outcome, cut the rate in that group by about half on its own. A system with no supported way to say "I found nothing" will be told something.

The idea is not new to agent systems. Attribution checking of exactly this form is standard practice in the evaluation of retrieval-augmented generation, the arrangement in which a model answers from passages fetched for the question rather than from what it remembers. There, a generated sentence is scored against whether the retrieved passages entail it, meaning the passages say enough on their own to make the sentence true. What changes in an orchestrator design is where the check has to sit. In a single retrieval-augmented call, the passages are still in context at generation time; here the worker's retrieved documents are gone by the time the parent has the claim, so the cache and the doc id are what make an otherwise ordinary check possible at all.

Verification of flagged claims costs what it costs. Scout dispatched a median of 1.7 of these per run at roughly 2,000 tokens and 6 seconds each, which is the price of turning a subset of single-sourced assertions into checked ones. The subset is chosen by the three conditions in is_load_bearing, so every claim outside it stays exactly as uncorroborated as it was.

None of this makes the answer true. Span validation is a check on the relationship between a claim and a document, which is a narrower guarantee than it sounds: a run in which every source is a competitor's marketing page produces claims that are all perfectly grounded and collectively misleading. Corroboration counting is worse than useless when two workers were sent to the same source, which is the connection back to chapter 2. Duplicated scopes produce claims that look independently corroborated and are one document counted twice, so the corroboration check has to compare doc_ids rather than worker counts. Two claims citing the same doc are one source, whatever the worker count says.

There is also a step the checks do not cover at all, and it is the last one. The synthesis model, handed a set of validated claims, writes prose — and prose can contain assertions no claim supports, either by combining two claims into an inference neither licenses or by inventing a connective statement to make a paragraph flow. Running the same span check over the final answer's sentences, rather than only over worker claims, closes that gap and is where Scout found its second-largest category of ungrounded statements. The aggregation boundary is not one boundary; it is one on the way in and one on the way out.

The fabricated 34% and the problem the next chapter opens on turn out to share a cause. In both cases the synthesis step made a decision about what to include and what to leave out, and in neither case did anything record the decision. A fabrication that gets in and a strong finding that gets dropped are the same unmeasured step failing in two directions, and until that step produces a record, every change to it is a change whose effect you cannot observe.