A default inclusion policy is judged by its exceptions

Here is the fill trace for a task on billing/pricing.py, from a builder that sorts candidates by retrieval score and takes them until the budget runs out.

The file that states what the change has to preserve was ranked thirteenth and did not make it in, while a deprecated experiment, a legacy reporting job and a UI formatter did. Nothing malfunctioned. An embedding similarity between a ticket and a test file is genuinely lower than between that ticket and three modules full of pricing vocabulary, because tests are written in the language of assertions and fixtures rather than the language of the feature. Score ordering answers what looks like the task, and the question a fill order has to answer is what does an edit here need in order to be correct.

Those come apart most sharply for the two things a coding agent needs most. Tests score low and carry the specification. Interfaces score low, since a stub file has almost no prose, and carry the boundary. Everything the previous chapters treated as a hazard — near-duplicates, deprecated implementations, other teams' conventions — scores high, because a near-duplicate is by construction the thing most similar to the target.

Filling by role#

The fix is to allocate the budget by role first and let retrieval compete for what is left. Each role gets a reservation, roles are filled in order, and unspent reservation flows down.

CEILING = 70_000          # first-turn fill, derived in chapter 1

ROLES = [
    ("target",        "full or skeleton by task type", 25_000),
    ("tests",         "suites naming the target's symbols", 15_000),
    ("interfaces",    "interface view of direct dependencies", 10_000),
    ("call_sites",    "spans + headers of callers of changed symbols", 8_000),
    ("retrieved",     "score-ranked remainder", 12_000),
]

def build_context(task: Task) -> list[Block]:
    blocks, spent = [], 0
    for role, _, reservation in ROLES:
        allowance = reservation + (CEILING - spent - remaining_reservations(role))
        chosen = collect(role, task, budget=max(0, min(allowance, CEILING - spent)))
        blocks.extend(chosen)
        spent += sum(b.tokens for b in chosen)
    return blocks

Run against the same task, that produces billing/pricing.py in full at 8.4k, tests/test_pricing.py in full at 6.3k, interface views of billing.money and the two packages pricing imports at 2.1k, the four call sites of compute_line_total as spans with their headers at 1.8k, and roughly 4k of score-ranked remainder — about 23k against a 70k ceiling, with the deprecated experiment and the legacy report absent because neither is a test, an interface, a call site, or high enough in the remainder to matter.

The test file earns its reservation twice over. It states the behaviour the change must preserve, which is the obvious half. The half that shows up in the diffs is that it also demonstrates how this repository builds fixtures and asserts on money — Money.from_minor_units(1999, "EUR") rather than Decimal("19.99"), and assert_money_equal rather than == — and almost every pricing change requires a new test, which will be written in whatever style the model has seen. Runs without the test file in context produce tests that pass and assert on floats, and a reviewer sends them back.

Four exceptions, and what each one is for#

A default that fits an ordinary application package will be wrong about the parts of a repository that are not ordinary application packages. Two of the four below turn on conventions worth stating in full: generated code is the set of modules a code generator writes from a schema, which here means the *_pb2.py that a protobuf compiler produces from a .proto message definition, and vendored code is a copy of a third-party library checked into this repository under vendor/ instead of pulled from a package registry at build time. Naming those explicitly is what makes the policy reviewable.

The generated-code rule is the one that surprises people who think of exclusion as a token-saving measure. Excluding a 40k _pb2.py does save tokens, and the reason it belongs in the policy is that including it invites a patch that edits generated output — which applies, compiles, passes tests, and is silently reverted the next time anyone runs the generator. Including the .proto instead makes the only correct edit the available one.

The tests-tree rule is a reminder that "include the tests" is underspecified in a repository where the tests are the larger half. Resolution by import graph — the suites that import or reference the symbols being changed — gives three or four files where a path glob, a wildcard pattern such as tests/** matched against file paths, gives four hundred.

The policy itself should live in the repository, versioned with it, because every rule above encodes a fact about this repository that will change.

# .agent/context-policy.yaml
version: 4
first_turn_ceiling: 70000        # recompute when the loop or prefix changes
roles: [target, tests, interfaces, call_sites, retrieved]

exclude:
  - glob: "**/*_pb2.py"
    include_instead: "proto/**/*.proto"
    reason: "generated; edits are reverted by the generator"
    owner: "@platform"
  - glob: "vendor/**"
    view: interface
    writable: false
    reason: "forked upstream; see vendor/PATCHES.md"
    owner: "@platform"
    review_by: "2026-11-01"

The review_by field is the part most policies lack. An exception written for a good reason in March is a guess about the repository by September, and dating it forces the question to be asked again rather than inherited.

Every rule owes an ablation#

A policy with five roles and four exceptions contains nine claims about what helps, and the honest position on any of them is that it helps until measured otherwise. The ablation is the same shape as the padding and permutation runs from the third chapter: hold the eval set fixed, remove one role's reservation, and read the change in first-attempt pass rate. Dropping the tests role on this repository cost 11 points and produced a characteristic failure — patches that were right and shipped with tests a reviewer rejected. Dropping the call-sites role cost 2 points, which says the 8k reservation is generous for this codebase and could be spent on the retrieved remainder. Dropping the interfaces role cost almost nothing on pass rate and changed the cross-boundary edit rate, which is a reminder that pass rate is not the only outcome a policy is accountable for.

Running nine ablations across 240 tasks is a few hundred agent runs, which is an afternoon of compute and cheaper than one production incident. What makes it worth institutionalising is that the numbers drift: the call-sites role that is worth 2 points today is worth more the month someone lands a large refactor, and a policy nobody re-measures is a policy that is being trusted rather than known.

The other instrument the policy needs is an escape hatch with logging. When the agent asks to read a path the policy excluded, the correct response is to allow it, record the request, and record whether the run went on to succeed. Those records are the highest-signal input to the next policy version, because they are cases where the model's judgment and the written rule disagreed and there is an outcome attached. In the first month here, the log was dominated by requests for two things — a shared conftest.py that the import-graph rule missed because pytest injects it implicitly, and the .proto files for services adjacent to the target — and both became rules in version 3 rather than staying arguments.

The argument for indexing everything#

The standing objection is that all of this is premature optimisation — index the repository, use the largest window available, and let the model pick, on the grounds that selection rules encode today's guesses and will be wrong tomorrow.

The premise is correct. Every rule above will go stale: the ceiling changes when someone adds four tools to the system prompt, the vendored fork gets upstreamed, the generated modules move behind a build step. A policy is a set of dated assumptions and should be treated as one.

The conclusion does not follow, for three reasons that the earlier chapters supply. An unrestricted index makes every deprecated, vendored and other-team implementation an available answer, and that is measurable rather than theoretical — recall 0.71 to 0.94 against first-attempt pass rate 58% to 41%, with the ablations showing most of the gap coming from the content of the extra files rather than from their position. A larger window fixes the truncation and eviction failures and leaves that one exactly where it was. And an agent given everything will edit anything: 12% of runs crossed a package boundary when the implementation was in context against 1.3% when it was not, and one of those crossings shipped a relaxed audience check that a green CI could not see.

The deeper answer is that trusting the model to choose is itself a policy. It is the policy with no stated boundaries, no record of what was considered, and no way to attribute a bad patch to a decision — when a run imports the legacy formatter, there is nothing to inspect, because the choice happened inside a forward pass. A written policy fails too, and its failures have a line number. That is the whole of the difference, and it is what makes the second version better than the first.

The fork nobody documented#

Which brings up the case that tests all of this. The vendored fork under vendor/httpclient carries a local patch — someone changed the retry backoff to be jittered, six years ago, in a commit whose message is fix flakiness, and it is not in vendor/PATCHES.md because vendor/PATCHES.md was created three years after that.

The policy behaves exactly as written. vendor/** gets an interface view, so the agent sees the public signature of HttpClient.request and none of the retry logic, and it writes a caller assuming the documented upstream behaviour of fixed backoff. Under load the two behaviours differ enough that the caller's own timeout is wrong, and it surfaces as an incident weeks later.

Every alternative fails here as well. Retrieval would have to rank a file nobody described as relevant. A whole-file default would put 30k tokens of vendored HTTP client in the window and, by the argument of the third chapter, hand the model a second implementation of everything it needs. A staff engineer's repository-wide index would return the fork alongside upstream and offer no way to tell which one is authoritative. The undocumented divergence is the problem, and no selection strategy recovers a fact that exists nowhere in the repository.

What the policy adds is a place to put the fix. The failure is attributable to one line in a versioned file, and the response is to make the divergence machine-readable — a PATCHES.md entry that the interface view appends as a note, or a lint rule that fails when a vendor/ file has commits not present upstream. The general form is that a policy converts an unwritten assumption into a written one, which is what makes it correctable; the assumptions that are still unwritten are the ones that will produce the next incident, and you find them by watching what the policy gets wrong.

That is the loop this topic ends on. Derive the ceiling from your own loop rather than from your window, order the fill by role, keep the eviction tiers where you can inspect them, and re-run the padding-and-permutation ablations on your own eval set when you change any of it — because the numbers in these chapters describe one monorepo, and the only ones that will govern your builder are the ones you measure.