Twenty thousand resting tokens before a single body loads

An internal platform team circulates a one-page proposal. They have been collecting skills from nine product groups for two quarters, they have 340 of them in a registry, and they want to publish all of them to every agent in the company, plus the roughly 660 more the groups have promised by Q3. The paragraph everyone in the review nods along to reads: skills use progressive disclosure, so a skill that is not relevant to a request costs nothing. Publishing the full catalogue is free. We should stop rationing.

The first half of that paragraph is true about bodies and bundled files. It is false about the tier that does the routing, and the arithmetic is one line: a thousand skills whose name and description come to twenty tokens each put 20,000 tokens into the prompt of every request the agent serves, before the model has read the user's question and before any body is fetched. That is not a peak or a worst case. It is the floor, and it is the same on the request that asks how to reset a staging database as on the request that needs four of the skills at once. That floor is the catalogue's resting cost: what a library charges a request before anything has been selected, fetched or used.

The half of the cost that appears on an invoice#

Twenty thousand tokens is a real number and it is worth pricing properly rather than dismissing or dramatising. The metadata block is an ideal cached prefix: it is byte-stable across requests, it sits at the front of the system prompt ahead of anything per-user, and it is comfortably above the minimum cacheable prefix. So after the first request it bills at cache-read rates, roughly a tenth of the input rate. On Claude Opus 5, at $5 per million input tokens, a cache read of 20,000 tokens costs about one cent. An agent serving two million requests a month is therefore paying on the order of $20,000 a month to have the catalogue resident, and paying it whether or not any skill is used.

Latency is the other visible line. A cache read is much cheaper than a fresh prefill but it is not free; on the platform team's own measurements, moving the resident block from 748 tokens to 20,000 added about 190 ms to median time to first token. For a chat surface that is tolerable. For an IDE plugin with a p95 budget, 190 ms of unconditional overhead on every keystroke-triggered request is a meaningful share of the budget, and it is spent identically on the 60% of requests that need no skill at all.

Caching also has a publication cadence attached to it that the monthly figure hides. The cached prefix is keyed on the exact bytes of the block, so adding one skill, or fixing a typo in one description, invalidates the cache for every agent in the company. The next request each of them serves pays the full input rate on all 20,000 tokens and re-writes the cache, which costs about ten cents rather than one and adds the full prefill latency instead of the cache-read latency. A registry that publishes once a week absorbs this without anyone noticing. A registry where nine product groups merge to a shared catalogue whenever they like, which is what the proposal describes, invalidates the prefix several times a day, and the resulting sawtooth in p95 latency will be blamed on the model provider long before anybody connects it to the merge queue.

Both numbers are computable from the repository before anybody ships anything, and computing them is the first thing to do when a proposal like this arrives. The count has to come from a real tokenizer against the real model — a character-count heuristic will be off by enough to change the conclusion on non-English descriptions and on descriptions dense with API identifiers.

import pathlib, frontmatter
from anthropic import Anthropic

client = Anthropic()

def resting_cost(skills_dir: str, model: str = "claude-opus-5") -> tuple[int, int]:
    """Total resident tokens, and skill count, for a directory of skills."""
    blocks = []
    for skill_md in pathlib.Path(skills_dir).glob("*/SKILL.md"):
        post = frontmatter.load(skill_md)
        # Only the frontmatter pair is resident; the body is tier two.
        blocks.append(f"{post['name']}: {post['description']}")

    joined = "\n".join(blocks)
    counted = client.messages.count_tokens(
        model=model,
        messages=[{"role": "user", "content": joined}],
    )
    return counted.input_tokens, len(blocks)

tokens, n = resting_cost("skills/")
print(f"{n} skills, {tokens} resident tokens, {tokens / n:.1f} per skill")
# 340 skills, 8114 resident tokens, 23.9 per skill

Run that in CI against the registry and the resting cost stops being an argument and becomes a number that shows up in a pull request. Twenty-four tokens per skill is what a two-sentence description costs in practice; the platform team's 340 skills were already sitting at 8,114 tokens, which nobody had noticed because nobody had measured it.

The half that does not#

The bill is the part of this that people argue about, and it is the smaller part. What the resident block also does is hand the model a discrimination problem whose difficulty scales with the size of the catalogue. Selecting a skill means reading a thousand short descriptions and picking the one that fits the request, from descriptions alone, before any body is available to disambiguate. At forty skills those descriptions are mostly about visibly different things. At a thousand, drawn from nine product groups who wrote them independently, they are not: there will be six skills about querying logs, four about generating a report from a warehouse table, three about resetting an environment, and each cluster's members differ in a clause that the request may not contain any signal about.

Three of the log-querying descriptions from the registry, written by three groups who had never read each other's, sat in the catalogue like this:

- name: query-service-logs
  description: Search application logs for a service over a time range, filter by
    level and request id, and summarise error patterns.
- name: log-search
  description: Run structured queries against the central log store. Use for
    investigating incidents and finding correlated errors across services.
- name: trace-lookup
  description: Retrieve logs and spans for a single request across services when
    you have a trace id or a request id.

A request that says find me the errors for checkout around 14:20 yesterday has a defensible claim on all three, and nothing in the request supplies the distinguishing signal, because the distinguishing signal is a piece of internal architecture the user does not know about. The model is not confused in an interesting way; it is being asked a question the descriptions do not contain the answer to. Multiply that by the four clusters like it, and the catalogue has built in a floor on how accurate selection can be, independent of how good the model is.

Here is what that did to the platform team's selection accuracy. The evaluation is 300 held-out real requests, each labelled with the skill a human considers correct, run against catalogues of increasing size built by adding real skills from the registry in the order they were published.

Between 40 skills and 1,000, the resident cost went up by a factor of 25 and the chance of the right skill being used went down by 26 percentage points. The two columns on the right are where the damage sits. A wrong selection loads a body full of confident, specific instructions for a task the user did not ask about, and the agent follows them. An absent selection means the agent answers from general knowledge, which for most questions produces something plausible and unremarkable.

Neither of those failures generates an error. The run completes, a response is returned, latency is normal, and the token count is if anything lower than a correct run because no body was fetched. Whatever dashboard the team watches will show a healthy service. The signal is a slow drift in how useful the answers are, distributed across every product group, attributable to nothing, and reported — if at all — as a vague sense that the assistant has got worse.

Where the arithmetic misleads people#

A team with 40 skills averaging 25 metadata tokens is paying 1,000 resident tokens. Moving to 400 skills takes that to 10,000. Against a million-token context window that is one percent, and the honest response to "what does this cost" is: in tokens, almost nothing. Every version of this argument that stops there reaches the same conclusion, and the conclusion does not follow from the premise, because the premise is measuring the wrong thing.

Ten thousand tokens of near-identical descriptions is not ten thousand tokens of volume. It is four hundred candidates the model has to tell apart on every single turn, with only a phrase each to go on. The thing that degrades is not the context budget; it is the decision. And because the decision's failure mode is a wrong or absent selection rather than a truncation or an error, the degradation lands in a place nobody is measuring. The token cost is on a dashboard. The selection quality is in the answers, and nobody reviews those.

This is also the reason that the two obvious mitigations do not help. Prompt caching addresses the money and part of the latency and does nothing at all for discrimination — the model still reads all 1,000 descriptions, cached or not. Buying a larger context window addresses neither; the descriptions were never close to filling the window. What is scarce here is not room, and treating it as a capacity problem produces solutions that measure well and change nothing.

It is worth being precise about what the resting cost is not, because it gets conflated with a neighbouring problem. This is not dilution in the sense the context-selection work covered, where a large block of loaded material makes the model's answer worse by burying the relevant part among near-neighbours. That happens after selection, to the material a skill brings in. Resting cost happens before selection, to the routing layer, and its victim is which skill gets chosen rather than what the chosen skill produces. A library can have a perfect resting cost and terrible dilution, or the reverse, and the fixes are different.

Naming the number#

The useful output of this arithmetic is a threshold written down in advance, because the alternative is that the library grows one justified skill at a time until it is too large and nobody can point at the request that broke it.

Two numbers are worth committing to. The first is a resting-token ceiling, chosen against the latency budget rather than the money: if the agent has a 3-second p95 and the metadata block adds roughly 10 ms per thousand tokens of cache read, then 10,000 resident tokens is a hundred milliseconds of unconditional overhead and is defensible, while 40,000 is four hundred and is not. The second, and the one that actually binds, is a floor on selection accuracy measured on a frozen set of labelled requests. The platform team settled on 90%: the catalogue may grow as long as the eval stays at or above 90% correct selection, and a proposed batch of skills that takes it below that is not merged.

On their numbers those two constraints bind at very different points. The token ceiling would have allowed roughly 420 skills. The accuracy floor stopped them at 130. That gap is the finding: the discrimination limit arrives at a third of the size the cost limit does, so a team reasoning only about tokens will conclude they have three times the headroom they have.

Which leaves an obvious problem for a platform team that genuinely has 400 capabilities their engineers use. A threshold tells them to stop at 130 and says nothing about the 270 skills that exist, work, and have users. Stopping is not an answer to that; the catalogue is the requirement, and something other than the catalogue has to change.