Forty-seven errors are usually one root cause and forty-six echoes

The ticket is a rename: RateCard.price becomes RateCard.unitPrice, because a second price field is arriving next sprint and the ambiguous name has already caused one incident. The agent edits the interface in packages/billing/src/types.ts, one line, and the loop runs tsc -b. Forty-seven errors come back across twelve files. The loop hands all forty-seven to the model, which does what anyone handed a list of forty-seven problems does: it works through them. The next attempt contains edits to twelve files and comes back with nineteen errors, most of which nobody had seen before. The attempt after that has twenty-three. On the fourth attempt the retry budget runs out and a human opens a diff with sixty-one changed lines in it, some of which are correct, some of which are as any, and one of which renamed a field on a different interface that happened to be called price.

Nothing in that run was a reasoning failure. The single edit was right, and it would have been right if a person had made it. What went wrong is that the loop presented forty-seven consequences of one cause as forty-seven independent problems, and then acted on the presentation.

Why the count is large and the cause is small#

A type error is a claim that two facts are inconsistent, and the checker reports it at every place where the inconsistency is observable. Renaming a field on a shared interface makes one fact change, and the number of observation points is the number of expressions anywhere in the program that read the old name. Those expressions live downstream in the module graph — the record of which file imports which, with an edge from every file to each file it depends on — so the errors distribute along dependency edges from the changed file:

Reading the graph bottom-up explains why the naive loop diverged. The fourteen errors in e2e/fixtures.ts are the furthest thing in the repository from the cause, and they are also the most numerous, so a model scanning the list weights them heavily. Every one of them says something like Property 'price' does not exist on type 'RateCard', and every one of them disappears the moment the call chain above it is consistent. When the agent patched the fixtures directly — adding casts, constructing objects with both field names, widening a parameter to Record<string, unknown> — it was making the file locally type-correct against an interface that was still mid-migration, and each of those edits was a new fact that the checker then found inconsistent with something else. That is where the nineteen novel errors came from on attempt two. Speculative patches made from downstream symptoms do not merely fail to help; they generate work.

The comparison between the two loops on the same task is the whole argument:

The right-hand column does more total attempts' worth of work with less total edited code, because every attempt is made against a diagnostic that is a cause at the moment it is read. The error count going up between attempt one and the state after it — 4 shown, 21 remaining — is not a regression; the checker simply reports more of the downstream once the upstream is consistent enough to get past the first failure. Counting total errors is a bad progress metric for this reason, and counting errors in the frontier file is a good one.

Dependency order is the main axis and not the only one, because some diagnostics invalidate others outright. A syntax error means the file did not parse, so every semantic diagnostic the checker produces about that file — and about anything importing it — is derived from a partial parse tree and may be nonsense; compilers know this, which is why rustc stops after parsing when it has to and why tsc's syntactic diagnostics are worth requesting separately from its semantic ones, as chapter 1's collector does. The rule that falls out is a severity ladder above the dependency sort: if any syntactic diagnostic exists, show only those; if any error exists, never show a warning alongside it. A loop that hands the agent a missing-brace error and thirty type errors in the same payload is offering thirty pieces of evidence generated from a file the compiler could not read.

Ordering by dependency, in about thirty lines#

The harness has two things the model does not: the module graph, and the diff. Both are cheap to get. tsc will emit the graph directly with --listFiles and --traceResolution, or you can read it from the build system's own dependency data; for Python, the import graph comes from importlib metadata or a static walk; for Rust, cargo metadata gives the crate graph and the module graph comes from the compiler's own resolution.

def frontier(diagnostics, deps, changed_files, max_files=2):
    """Return the diagnostics the agent should see this attempt.

    deps: {module: set(modules it imports)}  -- edges point at dependencies
    """
    by_file = collections.defaultdict(list)
    for d in diagnostics:
        by_file[d["path"]].append(d)

    # A file is a root candidate if none of its dependencies also has errors.
    failing = set(by_file)
    roots = [f for f in failing if not (deps.get(f, set()) & failing)]

    # The edit itself is the most likely cause, so it sorts first.
    roots.sort(key=lambda f: (f not in changed_files, -len(by_file[f])))

    selected, out = roots[:max_files], []
    for f in selected:
        out.extend(sorted(by_file[f], key=lambda d: (d["line"], d["column"])))
    return {
        "diagnostics": out,
        "shown": len(out),
        "suppressed": len(diagnostics) - len(out),
        "note": f"{len(diagnostics) - len(out)} further errors in "
                f"{len(failing) - len(selected)} downstream files were withheld; "
                f"they will be re-checked after this patch.",
    }

Two details in that function carry most of the value. Sorting changed_files first encodes the observation that the file the agent just edited is the likeliest cause, which is true often enough to be worth a tiebreak and cheap enough to be worth nothing when it is wrong. Telling the model that errors were withheld, and why, prevents the failure mode where the agent believes it has finished because the list it was handed is now empty; the note converts a silent truncation into a stated fact about the state of the build.

Cycles are the case the simple version gets wrong. In a repository where rates.ts and credits.ts import each other, neither is a root by the definition above and the frontier comes back empty. The standard repair is to condense strongly connected components — groups of files that can all reach each other through imports — into single nodes, using Tarjan's algorithm over the same graph and a dozen more lines, then treat a whole cycle as one frontier unit, which is also the honest description of what the engineer would do.

Why this is not a prompting problem#

The natural objection is that a capable model should be able to look at forty-seven errors, notice that forty-six of them mention price, infer that one interface changed, and go fix that. Add an instruction — identify the root cause first and ignore consequential errors — and the problem is solved without any pipeline work.

The instruction helps a little and cannot solve it, for a reason that has nothing to do with model capability. Deciding which of forty-seven errors is a cause requires knowing which files depend on which, and that fact is not in the diagnostics. Two errors reading Property 'price' does not exist in rates.ts and in e2e/fixtures.ts are textually identical; what makes one upstream of the other is an edge in the module graph that the model cannot see from the error list and can only guess at from the file paths. On this fixture the guess is available, because the names are suggestive and there are only twelve files. On a monorepo with forty packages, generated clients, barrel files that exist only to re-export the contents of a directory under one name, and a tsconfig path-mapping layer that rewrites import paths, the dependency direction between two files with similar names is not inferable from the names, and a wrong guess costs an attempt.

The deeper issue is that the instruction asks the model to compensate for input the harness chose to make misleading. Forty-seven items were placed in the context, and every one of them is a valid, precisely located, apparently actionable error. Asking the model to ignore forty-six of them is asking it to disbelieve most of what it was just told, on every attempt, using information it does not have. The harness does have that information, the computation is a topological sort — an ordering of the graph in which no file appears before the files it depends on — and it costs microseconds. When a fact is available to the harness and unavailable to the model, putting the decision in the prompt is choosing the worse of two implementations.

The same argument applies to the version of this the model genuinely cannot resolve: which diagnostics existed before the patch. That is a comparison against a run the model never saw, and it is a different problem from this one even though both arrive as noise in the same list. Cascades are a dependency problem, solved by ordering; standing warnings are an attribution problem, solved by differencing. Conflating them produces a pipeline that fixes half of the noise and leaves the agent still working through a list it should never have been shown.

Where dependency ordering stops helping#

Ordering assumes the errors have a small number of roots, which is true for the edit that motivated it and false for others. An agent that upgrades a dependency across a major version, or regenerates a client from a changed schema, or edits four unrelated functions in one patch, will produce diagnostics with many independent roots, and showing the first two files means twelve attempts to work through what should have been three. The adaptive form is to watch the count: if the number of failing root files does not fall between attempts, widen the frontier; if the same fingerprint reappears twice in a row — the same set of diagnostics, identified by their content rather than by their line numbers, in the way chapter 4 makes precise — the agent is stuck rather than progressing and the loop should stop rather than spend the remaining budget. Choosing that stopping rule sensibly needs a way to price attempts, which is coming.

There is also a category where no ordering helps because the root is not editable. When the frontier file is a generated client, a .d.ts declaration file shipped by a published package to describe the types of code compiled elsewhere, or vendored code copied in from another project, the correct patch is upstream of the repository — regenerate, upgrade, or change the schema — and a loop that keeps handing the agent errors in files it must not edit will get exactly what it asked for, which is a hand-edit to generated code that the next codegen run silently reverts. Marking directories as non-editable, and translating a frontier that lands in one of them into a different instruction, is a small amount of configuration that prevents a class of quietly wrong patches.

That leaves the second half of the noise problem untouched. Even with perfect dependency ordering, a repository with three hundred standing lint warnings hands the loop three hundred well-formed, precisely located problems on every single attempt, none of which the patch caused — which is the next chapter.