Relevance is the file that must change, not the one that discusses it

The ticket reads: the /export endpoint is missing rate limiting. Retrieval is now doing everything the previous chapters asked of it — symbol chunks with their enclosing scope, an import graph with resolved edges, route strings indexed as symbols — and the top five results come back as middleware/rate_limit.py, tests/test_rate_limit.py, docs/platform/rate-limits.md, docs/runbooks/throttling-incidents.md, and config/schema/limits.yaml. Every one of those files is about rate limiting. The file the patch has to change is routes/export.py, which contains no occurrence of "rate", "limit", "throttle", or "quota" in any of its 61 lines, and it ranked 34th.

Five results, none of them the patch#

The two files that matter sit side by side in the same directory and differ by one line.

# routes/reports.py
from middleware.rate_limit import rate_limited

@router.get("/reports/{report_id}")
@rate_limited(bucket="reports", rpm=60)
async def get_report(report_id: str, tenant: Tenant = Depends(current_tenant)):
    return await reports.render(tenant, report_id)

# routes/export.py
@router.get("/export")
async def start_export(fmt: str = "csv", tenant: Tenant = Depends(current_tenant)):
    job = await exports.enqueue(tenant, fmt)
    return {"job_id": job.id}

The patch is two lines: an import and a decorator. Producing it requires reading routes/export.py, reading enough of middleware/rate_limit.py to know that rate_limited takes a bucket name and a rate, reading config/schema/limits.yaml to add the export bucket so the decorator does not fail validation at startup, and ideally reading routes/reports.py to see the house style. Nothing in that list requires docs/runbooks/throttling-incidents.md, which scored fourth because it is 4,000 words of prose about rate limiting written in the same register as the ticket.

Sorting the candidates by what the change actually needs from them gives a different picture than sorting them by similarity.

The ranking is not uniformly wrong, which is what makes the failure hard to see from the metrics. Two of the top five are genuinely needed. What the ranking got right is the read set, and what it missed entirely is the edit set, and that split is not a coincidence: files that must be read are frequently the files that define and document a concept, which are exactly the files a topical scorer is good at finding, while files that must be edited are frequently the files that use or fail to use the concept, and using something leaves far less textual trace than defining it.

Two sets, and only one of them changes#

Relevance for a code task is best stated as an operational thing rather than a topical one: a file is relevant if a correct patch cannot be produced without it. That splits cleanly in two.

The edit set is the files whose bytes change. For this ticket that is routes/export.py, config/schema/limits.yaml, and tests/test_export.py. It is small, it is verifiable after the fact from the diff, and it is the set whose omission makes the change wrong rather than merely uninformed.

The read set is the files that must be understood for the edit to be correct. It includes middleware/rate_limit.py, because the decorator's parameter names are not guessable and getting them wrong is a TypeError at import; it includes the class or module invariants that constrain the edit, in the sense of the cache docstring from chapter two; and it includes at least one exemplar, a file that already does the thing correctly, because a repository's conventions are not written down anywhere and are learned by imitation.

The read set is larger and fuzzier than the edit set, and it tolerates error much better. An agent that reads one file too many spends tokens. An agent that reads one file too few writes a decorator with the wrong keyword argument, which the type checker catches. An agent that edits one file too few — the caller it never saw, the test that asserts on the old behaviour — produces a change that looks complete and is not, and that failure surfaces at build time in the best case and in production in the worst.

The exemplar deserves more weight than it usually gets, because a repository's conventions are the part of a codebase that exists nowhere in writing. Whether this service decorates handlers or wraps them in the router registration, whether the bucket name is a bare string or a constant from an enum, whether the rate is a literal or read from settings, whether the decorator goes above or below @router.get — none of that is in the middleware's docstring and all of it determines whether the patch reads as native or as foreign. routes/reports.py answers every one of those questions in five lines. A retriever that never surfaces a peer file forces the agent to invent a convention, and inventing a convention produces code that works and that a reviewer will ask to be rewritten.

This asymmetry is the reason a code retriever should not be tuned as though every retrieved file were equal. Recall on the edit set is the quantity that determines whether a patch can be right at all; recall on the read set determines whether it will be idiomatic. A retriever at 0.95 on read-set files and 0.55 on edit-set files will produce beautifully conventional patches that miss a caller, and an aggregate recall figure over the union of both sets will report something in between and explain nothing.

The task defined by a missing edge#

There is a reason routes/export.py is hard to find that goes beyond vocabulary. The ticket describes an absence. Retrieval systems locate things that exist — a term that appears, a symbol that is defined, an edge that was written down — and this task's defining property is that the edge from start_export to rate_limited is not in the source. No index of what is present will match on what is missing.

Two structural signals do reach it, and both work by triangulating rather than matching.

The route string is the first. The ticket says /export, and /export appears in the repository exactly once as a literal, in the @router.get decorator. Indexing route strings as symbols in the sense of the previous chapter makes that literal searchable, and a single exact match on a distinctive string is worth more than any amount of topical similarity — it is the difference between "this file is about the subject" and "this file is the subject". Distinctive literals in a task description, whether they are routes, error codes, config keys, event names, or a stack-frame path, are the highest-precision seeds available and are worth extracting from the task text explicitly before any embedding is computed.

The peer set is the second. Given the limiter's call sites, which the reverse call graph makes cheap to enumerate, you have a population: every route handler in the repository, some of which are decorated and some of which are not. The task's shape — X is missing Y — is answerable by retrieving the peers of the seed rather than its neighbours, letting the agent see six handlers that carry the decorator and one that does not. This is a genuinely different retrieval mode from similarity or expansion, it applies to a large family of tasks phrased as inconsistency or omission, and most retrievers do not implement it at all. Where it is not implemented, the practical substitute is to retrieve the sibling files in the same directory as any strong seed, which approximates the peer set on repositories with conventional layouts and does nothing useful on repositories without them.

Not every ticket is this hard#

The difficulty of a task, from a retrieval point of view, is largely determined by how much of the answer the ticket text already contains, and tickets vary enormously on that axis. Three rough classes cover most of what arrives.

A ticket carrying a stack trace is close to solved before retrieval starts. The frames name files and line numbers, and a frame path is an exact address rather than a description, so seeding on the innermost non-library frame and expanding from it recovers the edit set on the great majority of crash reports. The work here is parsing traces properly — recognising the format for each language in the repository, skipping vendor and standard-library frames, mapping a compiled or bundled path back to a source path through a sourcemap, the side file a bundler emits recording which original file and line each line of its output came from — and it is unglamorous work with a very high return.

A ticket naming a symbol, an error code, a config key, a route, or an event name is the middle case, and it is where the literal extraction described above does its work. The task text is prose, but it contains one or two tokens that are rare in the repository, and those tokens are worth more than the surrounding sentence.

A ticket that names nothing is the hard case, and the rate-limiting example is one: /export is the only address in it, and without route strings in the index there is no exact match at all. Tickets phrased entirely in product language — "enterprise customers say the download page feels unfair", "reconciliation is off by a few cents in Japan" — have no anchor, and this is where similarity, graph structure and history all have to carry the weight together. It is also where an honest system should be willing to say that it does not know and ask for a starting point, since the failure mode of guessing is a confident patch to a plausible neighbour.

Relevance belongs to the task, not to the file#

A tempting simplification is to compute a static importance for each file — PageRank over the import graph, which scores a file by how much of the graph's structure funnels through it and which is the same algorithm that once ranked web pages by their inbound links; change frequency; lines of code — and use it as a relevance prior. Those numbers have a use, mostly as a tie-break between candidates that score identically on everything else, and they are not relevance. middleware/rate_limit.py is central by any structural measure and is in the read set for this ticket, in the edit set for a ticket about changing the token bucket algorithm, and entirely irrelevant to a ticket about CSV encoding. The same file has a different answer for every task, which means relevance is a function of two arguments and any system that precomputes it on one of them has thrown away the other.

The corollary is that a code retriever cannot be evaluated by looking at its output and deciding whether the files seem related, since that judgement is exactly the topical judgement that produced the failing ranking. It has to be evaluated against tasks whose correct answer is known independently, and the only cheap source of independently known answers is changes that already happened, which is a later chapter's problem and a harder one than it appears.

What this chapter establishes is a target rather than a mechanism: the ordered list should put routes/export.py above docs/runbooks/throttling-incidents.md, and the reason it should is that one of them has to change and the other does not. Knowing that is not the same as building something that does it. The signals that would produce that order — the exact literal match, the graph distance from the limiter, the fact that one candidate is a definition site and another is a paragraph mentioning a term, the fact that routes/export.py and config/schema/limits.yaml have been edited together in eleven of the last fourteen commits that touched either — are individually weak, they disagree with each other on most candidates, and they have to be reduced to one order with a cutoff. That reduction is the next chapter.