Three thousand always-loaded tokens displace the task's own evidence

The file is 94 lines and 3,000 tokens. Eleven people contributed to it over eight months, and every one of them added their line in response to something that had gone wrong. Nobody wants to cut it, because cutting means telling a colleague that the thing that bit them last March is not worth four lines. The engineering manager has asked for a justification of what the file costs, and the fastest way to answer is arithmetic.

The team runs about 340 agent sessions a week. Each session loads 3,000 input tokens it would not otherwise load, so the file adds roughly 1.02 million input tokens a week. At $3 per million input tokens that is $3.06 a week, or about $12 a month; with prefix caching turning most of those into cache reads at $0.30 per million, it is closer to $1.30 a month. On a team whose total model spend runs into four figures monthly, the file is somewhere between a rounding error and a rounding error's rounding error.

That number is correct, and leading with it ends the conversation at the wrong place. It measures the one cost of a resident config file that is genuinely negligible, and it will be re-derived by the next person who wants to add forty lines, with the same conclusion. Two costs that matter do not appear on an invoice at all. The first is displacement: the share of the working context the file holds at the moment the agent is deepest into a task, competing with the file contents, tool output and history that the task actually depends on. The second is that 94 lines of standing text are 94 claims about the repository, none of which anything in the repository verifies, and the count of unverified claims grows every month while the bill does not move. This chapter takes the first; the next takes the second.

Refusing to lead with the token bill is not the same as refusing to compute it. Compute it, put it in the third paragraph of whatever you write, and let it be the small number it is, because an audience that suspects you are hiding the cheap answer will not follow you to the expensive one.

What the file holds while the agent is working#

Follow one long task: adding idempotency keys to the invoice endpoint, which touched six files across two packages and ran 41 turns. It ran long enough for the harness — the agent tool that assembles the prompt, calls the model and executes its tool calls — to compact the session on the way, which is the step where the accumulated turn-by-turn history is replaced with a shorter summary of itself so that the context does not grow without bound. Sampling the context at three points, either side of that step, shows what the file is actually competing with.

The middle row is the one that makes the file look harmless, and it is the row least connected to how the session goes wrong. The interesting rows are the first and the third. At turn 6, a fifth of everything the model is attending to is standing text written for no particular task, at the moment when the agent is deciding where to look and what the task even is. At turn 24, the config file is 3,000 tokens of full-fidelity text sitting next to a 4,800-token lossy summary of the twenty-two turns in which the actual work happened.

That third row is the shape of the problem. A resident config file is the only thing in the context that never degrades: it is re-injected verbatim after every compaction, while the record of what the agent read, tried, and learned is compressed to a paragraph. The file's priority relative to the task's own evidence therefore rises exactly as the session gets long and difficult, which is the opposite of what anyone designing the system would have chosen. Nothing overflowed and nothing errored; the 200k-token context window, which is the total amount of text the model can be shown on any single call, was never close to full at turn 22. The harm is a reallocation inside a budget that was never exhausted.

There is a second effect that shows up before compaction, and it is easier to measure than to reason about. Running the previous chapter's paired design with three arms on the import-ban fixture — the real 3,000-token file, and a 400-token file containing only the four entries with fixtures behind them — the ban was respected in 3/5 runs with the full file and 5/5 with the trimmed one. Nothing was removed that bore on the ban; ninety other lines of standing instruction were removed from around it. A long list of low-value instructions makes the high-value ones harder to pick out, which is a cost paid by the entries you care about most.

Latency, and what caching really buys#

The prefill side is measurable and small. Across 40 sessions on the same hardware and network, median time to first token was 1.34 s with the file and 1.19 s without, so the file costs about 150 ms at the start of every session when its tokens are not cached. Prefix caching removes most of that, because the file sits in a stable prefix ahead of anything task-specific, and a cache read is both cheaper and faster to prefill than a fresh one.

The first call of a session that writes the cache reports it plainly:

{
  "model": "claude-sonnet-4-6",
  "usage": {
    "input_tokens": 14,
    "cache_creation_input_tokens": 3218,
    "cache_read_input_tokens": 0,
    "output_tokens": 412
  }
}

Every subsequent call in that session, and every session that starts while the cache entry is still alive, reports the other shape:

{
  "usage": {
    "input_tokens": 14,
    "cache_creation_input_tokens": 0,
    "cache_read_input_tokens": 3218,
    "output_tokens": 388
  }
}

The 3,218 rather than 3,000 is the file plus the wrapper the harness puts around it, and it is worth knowing which number you are budgeting against. A cache write is billed at 1.25× the base input rate and a cache read at 0.1×, with a five-minute sliding time-to-live (TTL) by default, so a file that nobody edits is close to free after the first call of the day and a file that changes is not. Editing one line near the top invalidates the cached prefix from that point onward, which means every session started after the edit pays a write again.

Follow that through and it stays small. If the file is edited on 12 of 20 working days, the additional cost is one cache write per invalidation per node, on the order of pennies, and one full prefill of 150 ms for the first session after each edit. The honest report of the caching story is that it makes an already-negligible bill slightly more negligible, and that the invalidation cliff people worry about is not where the cost lives either.

Establishing whether the file is implicated#

The team's actual complaint is that on long sessions the agent starts losing track of things it was told earlier — a constraint stated at turn 3 that it violates at turn 30, a file it read twice. That symptom has many possible causes, and asserting that the config file is one of them is exactly the move the previous chapter ruled out.

The paired design handles this with a third arm. Take one long fixture task, at least 35 turns on a frozen commit, and run three variants five times each: the real 3,000-token file, a 400-token trimmed file with the load-bearing entries only, and a 3,000-token placebo consisting of the trimmed entries plus 2,600 tokens of true but inert prose about the service's history. The placebo is the arm that makes the experiment worth running, because it separates two hypotheses that otherwise travel together.

Reading the results is mechanical. If the symptom tracks length — present in the real file and the placebo, absent in the trimmed one — the cause is displacement, and the remedy is a smaller file regardless of which lines are good. If it tracks content — present in the real file, absent in both the placebo and the trimmed one — then something specific in those ninety lines is doing damage, and you have narrowed the search to content rather than size. On this repository the symptom appeared in 5/5 real runs, 4/5 placebo runs and 1/5 trimmed runs, which is the displacement pattern, and it made the case for a budget without needing anyone to argue about individual lines.

A size rule that means something#

An absolute cap in tokens is easy to state and easy to game, since a team that is told 1,500 tokens will write 1,500 tokens of compressed prose that says the same inert things. The useful rule has two halves, one about the file and one about entries.

The file's half is a budget expressed against the context that survives compaction rather than against the model's window. The window is 200k and irrelevant; what matters is the roughly 20k the session is rebuilt from, and a resident file should hold under 5% of it. That gives about 1,000 tokens for this setup, and it is a number you can check without judgment:

#!/usr/bin/env bash
# scripts/check-config-budget.sh — run in CI on every PR
BUDGET=1000
tokens=$(pnpm tsx scripts/count-tokens.ts CLAUDE.md)
if [ "$tokens" -gt "$BUDGET" ]; then
  echo "CLAUDE.md is ${tokens} tokens, budget is ${BUDGET}."
  echo "Adding an entry requires removing one, or raising the budget in this file."
  exit 1
fi
echo "CLAUDE.md: ${tokens}/${BUDGET} tokens"

The entry's half is the standard from the previous chapter: an entry may be added when someone can name the task on which it changes a decision. The budget is what forces that standard to be applied, because a full budget converts every addition into a comparison between the new entry and the weakest existing one, and a comparison is a conversation people can actually have. Raising the budget stays available, and requiring it to be a line-change in a checked-in script means raising it is a decision somebody makes rather than a drift nobody notices.

Two things this rule does not do. It does not make the file good — 990 tokens of well-compressed platitudes passes the check and changes nothing, so the budget is a backstop against growth rather than a quality standard. And it does not survive the obvious workaround, which is to move the bulk of the file into linked or imported files. Whether that helps depends entirely on the tool: if the linked content is inlined into the prompt at session start, the tokens are still resident and the budget check is now measuring the wrong file, whereas if the tool loads linked files only when the agent chooses to read them, the split is a real reduction. Check which one your tool does before reorganising, and point the budget script at whatever is actually loaded.

The budget and the attribution test between them assume something neither of them checks. Both ask whether an entry earns its place if it is true; neither asks whether it is true. An entry can be precise, executable, attributable, well within budget, and quietly wrong about the repository — and the next chapter is about how long that survives, which turns out to be much longer than anyone would guess.