A decomposition is an artefact to check before it is dispatched

A decomposition is the list of subtasks an orchestrator produces when it reads a goal: one line per piece of work, each destined for a separate model call with its own tools and its own fresh context. It is the first artefact any orchestrator–worker run creates and the only one that exists before anything expensive happens. Every worker's instructions come from it, the synthesis step can only see material the workers were sent to find, and no downstream component can repair a question that was never asked. That makes its properties worth stating precisely rather than hoping for.

The two properties that matter are coverage, meaning the subtasks between them address the whole goal, and disjointness, meaning no two subtasks address the same thing. Scout — an internal competitive-research assistant at a company selling analytics software to enterprises — failed both on the run that opened this topic, when a sales director asked how a competitor called Vantage had changed enterprise pricing over the previous eighteen months. Neither failure was visible in the answer, and both were visible in the plan.

The six scopes, and what they cost#

Here is the decomposition Scout produced for the Vantage question, printed from the trace as it was handed to the dispatcher:

st_1  "Research Vantage's current enterprise pricing"
st_2  "Find how Vantage's pricing has changed over the past 18 months"
st_3  "Investigate Vantage's pricing page history and packaging changes"
st_4  "Gather customer sentiment about Vantage pricing from review sites"
st_5  "Search CRM notes for competitive intelligence on Vantage deals"
st_6  "Summarise implications for our renewal motion"

The run cost 41,300 tokens across the six workers. Of that, st_1, st_2 and st_3 consumed 19,400 tokens between them and returned material that a reviewer, laying the three outputs side by side, could compress into a little more than one worker's worth of distinct content: all three found the archived pricing pages, two of them found the same March press release, and st_2 and st_3 both quoted the same paragraph about packaging. Meanwhile the sales director's second question — whether Vantage had changed its channel-partner discount schedule, which is what actually decides how a reseller-sourced renewal gets defended — was assigned to nobody. It appears nowhere in the six scopes, and consequently nowhere in the answer.

Notice that those are the two failures named above, in the order the list produces them. Three subtasks covering one question is disjointness failing; one question covered by no subtask is coverage failing. They are not independent accidents. An orchestrator writing a fixed-length list of research tasks has a budget of lines to spend, and every line it spends restating a question it has already asked is a line it does not spend on a question it has not.

No worker misbehaved. Each of the six was handed an instruction and produced a competent response to it. The defect is entirely upstream: it is a property of the list above, and it was fully determined at the moment that list was printed, before a single worker was dispatched. That is the useful thing about it, because an artefact that is fully determined before anything expensive happens is an artefact you can check.

Duplication in the plan becomes confidence in the answer#

The wasted tokens are the least of it, and the reason is a chain worth walking one link at a time rather than summarising. Three workers are sent to overlapping ground. Each of them independently finds the same March press release, because it is the most prominent document about the subject they were all given. Each writes it up in its own words, so three outputs arrive asserting the same thing in three phrasings. The synthesis step then reads those three outputs and counts how many of its inputs support the claim. It finds three, and three independent sources agreeing is exactly the pattern that should read as strong corroboration.

Except that there was one source, counted three times. The synthesiser cannot see this, and the reason it cannot is structural rather than an oversight in how it was prompted. By the time the outputs reach it, nothing in them records that st_1, st_2 and st_3 had been sent to the same place; a worker's output carries what it found, not the scope it was competing with. Duplication in the plan turns into false confidence in the answer, which is a more expensive failure than the 19,400 tokens, and it is invisible downstream for the same reason the wasted tokens are invisible: the evidence of it was in the plan, and the plan is not in the prompt.

Prose scopes cannot be checked, so stop emitting prose scopes#

The three overlapping subtasks are indistinguishable from good ones by inspection of their text. "Research Vantage's current enterprise pricing" and "Investigate Vantage's pricing page history and packaging changes" are both reasonable English sentences, and deciding whether they overlap requires understanding what a worker will do with each — which is exactly the thing you do not know until the worker has done it. There is no string operation over that list that detects the problem, because the problem is not in the strings.

The first thing most teams try is an instruction. The decomposition prompt gains a line saying the subtasks must not overlap, and together they must fully cover the goal, which is a true statement of the requirement and does move the numbers a little. Scout's overlap rate — measured as the fraction of runs in which two workers cited a common source — went from 61% to 44% after that sentence was added, and stayed there.

That plateau is the thing to understand, because the instinct on seeing 61% fall to 44% is to write a better sentence, and a better sentence does not exist. The model is being asked to assert a property of a list it is composing as it writes. When it emits the third subtask it has no mechanism for checking that subtask against the two above it beyond re-reading its own output, and nothing in its training makes such a check happen reliably or makes its failure visible. So it says the scopes are disjoint because it was told to make them disjoint, and the saying is free. The residual 44% is the rate at which a free assertion happens to be false.

The stumble here is worth naming plainly, because it recurs in every chapter of this topic in a different costume. A quantity was held constant and treated as a defence — the instruction, present in every prompt, correct in what it asks — while the property that decides the outcome varied freely underneath it, that property being whether anything ever checks the assertion. An instruction that cannot be falsified is not a control. What is missing is not a better instruction but a representation in which the claim is falsifiable.

Two claims about identifiers instead of two claims about prose#

The change is to make the orchestrator commit to the two properties in a form that code can evaluate. The orchestrator is required to emit, alongside the subtasks, the set of sub-questions it believes the goal decomposes into, and then to assign each sub-question to exactly one subtask.

That indirection is what buys everything else, so it is worth stating what happened to each property. Coverage was "the subtasks address the whole goal", a claim about English; it becomes "every sub-question identifier appears in some subtask's list", a claim about set membership. Disjointness was "no two subtasks address the same thing"; it becomes "no sub-question identifier appears in two subtasks' lists". Both are now statements about identifiers rather than about prose, and both can be wrong in ways a validator can see.

{
  "goal_id": "g_4471",
  "sub_questions": [
    {"id": "q1", "text": "What were Vantage's published enterprise list prices at each point in the 18-month window?"},
    {"id": "q2", "text": "What packaging changes accompanied any price change (add-ons folded in or split out)?"},
    {"id": "q3", "text": "Which customer segments were affected, and when did changes take effect for each?"},
    {"id": "q4", "text": "Did Vantage change channel-partner discount schedules in the window?"},
    {"id": "q5", "text": "How are existing Vantage customers reacting to the changes?"}
  ],
  "subtasks": [
    {"id": "st_1", "question_ids": ["q1", "q2"],
     "sources": ["web.archive.org:vantage.com/pricing", "vantage.com/newsroom"],
     "excludes": ["g2.com", "crm"],
     "budget": {"max_sources": 8, "max_output_tokens": 1200}},
    {"id": "st_2", "question_ids": ["q3"],
     "sources": ["crm:competitive_notes"],
     "excludes": ["web.archive.org", "g2.com"],
     "budget": {"max_sources": 25, "max_output_tokens": 1200}},
    {"id": "st_3", "question_ids": ["q4"],
     "sources": ["vantage.com/partners", "web.archive.org:vantage.com/partners"],
     "excludes": ["crm"],
     "budget": {"max_sources": 6, "max_output_tokens": 900}},
    {"id": "st_4", "question_ids": ["q5"],
     "sources": ["g2.com", "reddit:r/analytics"],
     "excludes": ["crm", "vantage.com"],
     "budget": {"max_sources": 15, "max_output_tokens": 1000}}
  ]
}

Four subtasks instead of six, and q4 exists because the orchestrator was asked to enumerate the goal's sub-questions as a separate act before assigning them. Enumerating questions and enumerating work are different operations, and merging them is how q4 disappeared the first time: the original prompt asked for a list of research tasks, so the model produced a list of research tasks, and a question that nobody was going to research simply never had a reason to be written down.

Each subtask also carries three fields the prose scopes had no room for. sources names where the worker may read, excludes names where it may not, and budget caps how many sources it may consult and how long its reply may be. Those turn a scope from a description into a set of permissions, which matters for a reason that arrives two sections below.

The validator that runs before dispatch#

class DecompositionError(Exception):
    def __init__(self, violations: list[str]) -> None:
        super().__init__("; ".join(violations))
        self.violations = violations

MAX_OUTPUT_TOKENS = 1500
MAX_SOURCES = 30

def validate(plan: Plan) -> None:
    violations: list[str] = []
    question_ids = {q.id for q in plan.sub_questions}

    assigned: dict[str, list[str]] = {qid: [] for qid in question_ids}
    for st in plan.subtasks:
        for qid in st.question_ids:
            if qid not in question_ids:
                violations.append(f"{st.id} claims unknown question {qid}")
            else:
                assigned[qid].append(st.id)

    for qid, owners in assigned.items():
        if not owners:
            violations.append(f"{qid} assigned to nobody (coverage)")
        elif len(owners) > 1:
            violations.append(f"{qid} assigned to {owners} (disjointness)")

    seen_sources: dict[str, str] = {}
    for st in plan.subtasks:
        for src in st.sources:
            if src in seen_sources:
                violations.append(
                    f"{st.id} and {seen_sources[src]} both read {src}")
            seen_sources[src] = st.id
        if st.budget.max_output_tokens > MAX_OUTPUT_TOKENS:
            violations.append(f"{st.id} output budget too large")
        if st.budget.max_sources > MAX_SOURCES:
            violations.append(f"{st.id} source budget too large")

    if violations:
        raise DecompositionError(violations)

The two properties are enforced by the same loop read in two directions. assigned starts as one empty list per sub-question identifier, and every subtask appends its own id to the list of each question it claims. When the loop finishes, that dictionary holds the whole of both claims: a question whose list is empty is a question nobody owns, which is coverage failing, and a question whose list has more than one entry is a question two subtasks own, which is disjointness failing. Neither check needs to understand what a question means. They are conservation laws over identifiers — every identifier in, exactly one owner out.

The source check is the part that catches the failure the question-level check misses. Two subtasks can own different sub-questions and still be told to read the same archived pricing pages, in which case they will both spend their budget on the same documents and return overlapping material even though the plan is formally disjoint at the question level. Requiring the union of sources to be a partition too — a partition being a split in which every source is read by exactly one subtask, none shared and none left out — is stricter than strictly necessary, since occasionally one source genuinely bears on two questions, and the stricter rule is worth keeping, with an explicit shared_sources field for the exceptions, so that sharing a source is something the orchestrator has to declare rather than something it can drift into.

None of these checks require a model. They are ordinary set operations over identifiers, they take microseconds, and they run before any worker is dispatched, which means the cost of catching a bad plan is the cost of the planning call and nothing else. The original six-subtask plan would have failed on three counts: q4 unassigned, web.archive.org:vantage.com/pricing claimed by st_1, st_2 and st_3, and no budgets at all.

What passing this validator licenses is worth stating exactly, because it is narrower than it feels. It licenses one claim: the plan is well-formed against the question set the orchestrator itself wrote down. The residual is everything else. A conservation law conserves whatever you pour into it, and these laws are applied to the orchestrator's own enumeration, so a goal the model misunderstood produces a question set that is wrong and a plan over it that is flawless.

Feeding a violation back to the planner#

A violation is not an error to log. It is a signal to re-plan, and the useful part is that the violations list is precise enough to feed back to the orchestrator as input.

def plan_with_validation(goal: str, attempts: int = 3) -> Plan:
    feedback = ""
    for attempt in range(attempts):
        plan = orchestrator.decompose(goal, feedback=feedback)
        try:
            validate(plan)
            return plan
        except DecompositionError as err:
            feedback = (
                "Your previous decomposition was rejected for these reasons:\n"
                + "\n".join(f"- {v}" for v in err.violations)
                + "\nEmit a corrected decomposition."
            )
    raise PlanningFailure(goal, feedback)

The specificity of the feedback is what makes the loop converge. A retry that says only your plan was invalid, try again re-rolls the same distribution and lands on a similar plan, because nothing in the second attempt's context distinguishes it from the first. A retry that says q4 assigned to nobody and st_1 and st_3 both read web.archive.org:vantage.com/pricing names two edits, and the model makes them. This is the same discipline as a compiler error carrying a line number, and the cost of getting it wrong is the same: a vague diagnostic turns a two-attempt fix into an attempt budget spent on guesses.

Two things in that function are decisions rather than requirements, and they are worth separating. Re-planning at all is a requirement of having a check — there is no other sensible thing to do with a rejected plan. The number of attempts is a decision: it bounds how long a run may sit in planning before it gives up, and somebody chose it and could choose otherwise. So is what happens after the budget is spent, which in Scout is a PlanningFailure rather than a dispatch of the last plan tried.

The economics of checking early are worth stating in tokens. A Scout decomposition call costs about 1,900 tokens in and 700 out; a full dispatch of four workers plus synthesis runs 30,000 to 45,000. Re-planning twice therefore costs roughly a fifteenth of a wasted dispatch, so the loop pays for itself if it prevents one bad run in fifteen, and it was preventing closer to one in four. The asymmetry only gets steeper as worker counts rise, because the planning call stays a fixed small cost while the dispatch it guards grows with the number of subtasks.

The budget fields do work that is easy to overlook. A scope like "research Vantage's go-to-market" is not merely vague, it is unbounded, and an unbounded scope produces a worker prompt that grows until the worker is reading forty pages and trying to compress them into an output that no longer fits its own reply. Bad boundaries and oversized prompts are the same defect seen at two moments: at planning time it looks like a scope you cannot write an exclusion list for, and at execution time it looks like a worker whose output is truncated or whose recall collapses because the relevant paragraph was in the middle of a 60,000-token context. Forcing every subtask to declare a source cap and an output cap makes the first symptom visible while it is still cheap.

The right-hand column is where that table earns its place, and the gap in it falls between the fourth row and the fifth. Four symptoms are caught for the price of a planning call. The fifth is caught, if at all, after every worker has run and spent its budget, which is between one and two orders of magnitude further along in cost.

That gap is not an argument for pushing the fifth row upward. The last row is deliberate. Validation removes the disagreements that the plan manufactured by handing the same ground to two workers, and it leaves the disagreements that are real — two genuinely different sources saying different things about the same question, which is information rather than waste.

A well-formed plan can decompose the wrong goal#

Passing the validator says the plan is well-formed. It says nothing about whether the sub-questions are the right sub-questions, and the orchestrator can produce a perfectly disjoint, perfectly covering decomposition of the wrong goal. If the model reads "what does it mean for our renewal motion" as a request for general competitive background, it will enumerate five sub-questions about competitive background, assign them cleanly, and pass every check. The checks are conservation laws over whatever the orchestrator decided to enumerate; they do not audit the enumeration against the goal, and there is no cheap code-level check that does.

What you can do is sample. Take forty representative goals, have two people who know the domain write the sub-question list each goal deserves, and score the orchestrator's enumeration against those lists for recall — how many of the human sub-questions appear in some form — and for precision, meaning how many of its own questions the humans considered irrelevant.

Scout's enumeration ran at 0.78 recall against that reference set, and where the misses fell matters more than the number. They clustered: questions requiring domain knowledge about how renewals are actually defended were the ones it did not think to ask, while questions answerable from public sources it enumerated reliably. That is a fixable gap, addressed by putting the recurring sub-question patterns in the decomposition prompt as examples, and it is only visible because someone built a reference set. Without one, an enumeration failure is indistinguishable from an answer that happens to be short.

There is also a stronger limitation, and it is not a gap in the validator. The whole approach assumes the sub-questions are knowable at planning time. Coverage is checkable because the orchestrator can state the question set before dispatch; disjointness is checkable because those questions can be allocated before any of them are answered.

Both assumptions fail for a class of goal where what you need to ask second is determined by what you find first. For those goals, forcing a decomposition through this validator produces a plan that passes every check and is wrong in a way no check can see — and the symptoms it produces are the duplicated and missing scopes this chapter opened on, arriving from a cause no amount of care with the decomposition prompt can remove.