A bundled reference is a fork that nothing recompiles

A new engineer asked the agent how to authenticate a service-to-service call against the internal platform API, got a clear answer with a working-looking code sample, and spent two days on an integration that returned 400 unsupported_grant_type on every request. The flow the agent described — POST /oauth/token with grant_type=password and the service account's credentials in the body, the password grant, in which a caller trades a username and password directly for an access token rather than going through a redirect — was removed in February, eleven months after it was deprecated. The platform docs in the repository say so at the top of the page, in bold, with a migration guide underneath. The agent had never read them.

What it had read was reference/auth.md, bundled inside the platform-api skill, copied out of those same docs in November 2025 by someone doing exactly the right thing according to every instinct about making an agent self-contained. On the day it was written the file was accurate. Nothing about it has changed since, and nothing about it will, because a markdown file copied into a skill directory has no relationship to the thing it was copied from.

Copying created a branch, and there is no merge#

The mental model that fits is a fork. reference/auth.md is a branch of docs/platform/auth.md taken at one commit, and the two have been diverging ever since with no process watching. Ordinary forks are survivable because something eventually forces the comparison — a merge conflict, a failing test, a dependency bump. This one has nothing. No compiler resolves it, no test imports it, no linter reads it, and the file it forked does not know it exists.

Vendored dependencies are the closer comparison, and they come out looking sophisticated. When a Go module is vendored, the version is recorded, go mod verify checks the hashes, and a bot opens a pull request when upstream moves. A copied reference file records nothing: not the source path, not the commit it was taken at, not who copied it, not when. The information needed to detect the drift was thrown away at the moment of copying, and the copy is byte-for-byte as credible as the original when it lands in context.

The delivery mechanism makes it worse. Because reference files load only when the skill fires and the model decides that particular file is relevant, the stale content is not sitting in front of anyone. It appears inside a specific answer to a specific question, phrased as fact, and then disappears. A stale constant in a codebase gets read by every engineer who opens the file; a stale reference in a skill gets read by nobody until it is quoted back to a user who has no way to know its provenance.

Tracing one run end to end#

The way to see any of this is to reconstruct what actually entered context on the run that produced the wrong answer, rather than reasoning about what should have. Most runtimes will tell you if you ask them; the useful form is a manifest with sizes and timestamps.

{
  "session": "s_9f31c2",
  "request": "how do I authenticate a service-to-service call to the platform API",
  "loaded": [
    {"path": "skills/platform-api/SKILL.md",            "tokens": 1180,
     "reason": "description match", "mtime": "2026-01-08"},
    {"path": "skills/platform-api/reference/auth.md",   "tokens": 3240,
     "reason": "referenced by SKILL.md line 22", "mtime": "2025-11-14"},
    {"path": "skills/platform-api/reference/rate-limits.md", "tokens": 2610,
     "reason": "referenced by SKILL.md line 22", "mtime": "2025-11-14"},
    {"path": "skills/db-migrations/SKILL.md",           "tokens": 0,
     "reason": "not matched"}
  ],
  "scripts_invoked": [],
  "answer_cites": ["reference/auth.md#password-grant"]
}

Three facts fall out of it. The answer cites a file whose modification date is five months before the deprecation, which is the defect. rate-limits.md was pulled in as well and was never used, because SKILL.md line 22 says "see the files in reference/ for details" and that sentence loads both — 2,610 tokens spent on nothing, on every run of this skill. And no script ran, which is fine here and would not have been if the skill claimed a validation step.

Running that trace on three or four representative requests per skill, once, is the cheapest audit in this whole topic. It surfaces stale files, files that load together when only one was wanted, references that nothing ever reaches, and the token cost of the skill firing at all — none of which is visible in the answer, which is the entire problem. The platform-api skill costs 7,030 tokens every time it loads, of which the run above needed about 4,400.

Four ways to carry reference material#

The copy is one option among several, and it is chosen by default rather than on merit.

Most teams should be on the second column and are on the first. If the agent already has filesystem access to the repository — which it does, because it is writing code in it — then SKILL.md can say "the authentication flows are documented in docs/platform/auth.md; read it before answering" and the fork never exists. The cost is a dependency on the path, and a rename gives you a file-not-found in the trace, which is the loud failure the copy denied you.

The third column is worth the effort where the source is somewhere the agent cannot reach, an internal wiki or another repository. A step in CI copies the file and the build fails when the source has moved and nobody has re-run it, which converts an invisible drift into a red pipeline. The fourth belongs to material that changes faster than any build — the current on-call rota, this quarter's rate limits — and there the right answer is usually a tool rather than a reference file, because you are asking for live data and that is what tools do.

Making a copy that announces its own age#

Sometimes the copy is genuinely the right call: the source is a vendor's documentation site, or a PDF, or a Confluence page that the agent has no credentials for. Then the job is to make staleness detectable rather than to pretend it will not happen.

---
source: docs/platform/auth.md
source_commit: 4f1c9ab
copied: 2025-11-14
copied_by: @rmoreno
review_by: 2026-02-14
sha256_of_source: 9c1e...a7
---

## Platform API authentication
...

That header is inert on its own. What gives it teeth is eleven lines in CI that walk every reference/*.md with a source field, hash the current source, and fail if it differs.

#!/usr/bin/env bash
set -euo pipefail
status=0
for f in skills/*/reference/*.md; do
  src=$(yq '.source // ""' "$f"); [ -z "$src" ] && continue
  want=$(yq '.sha256_of_source' "$f")
  have=$(sha256sum "$src" | cut -d' ' -f1)
  if [ "$want" != "$have" ]; then
    echo "stale: $f (source $src changed since $(yq '.copied' "$f"))"
    status=1
  fi
done
exit $status

On the repository this chapter is drawn from, adding that check turned up nine stale references across six skills on the first run, the oldest copied fourteen months earlier. Two of them were describing behaviour that no longer existed anywhere in the platform. The check does not tell you what changed or whether it matters — someone has to read the diff and either update the copy or bump the hash — and it costs about four seconds in the pipeline.

Where there is no upstream at all, the calculus changes. A reference file describing the team's own conventions, tribal knowledge that has never been written down anywhere else, is not a fork of anything; it is the source of truth, and the risk is not divergence but abandonment. Those files get a named owner and a review date in the frontmatter, and the review date is enforced by a person, because there is no hash to compare against.

The reference that loads sometimes#

The manifest above is from one run, and running the same request twice does not guarantee the same manifest. Reference files are read at the model's discretion, part-way through a session that already contains whatever came before, and the decision to open a 3,240-token file is made against a context budget that a long session has already spent.

Two traces of the same question, one from a fresh session and one from turn 34 of a session that had been refactoring a client library for an hour, differed in exactly that way. The fresh session read auth.md in full and quoted it. The long session read SKILL.md, did not open any reference file, and answered from the model's general knowledge of OAuth — which produced a client-credentials flow that was closer to correct than the deprecated one in the stale file, and was still not the platform's actual flow, and cited nothing. Same skill, same request, two different mechanisms behind the answer, and no signal in either answer about which had happened.

This is the part of on-demand loading that surprises people who think of a skill as a document that either applies or does not. What applies is SKILL.md; everything it points at is a suggestion the model may decline under pressure. The practical consequences are that anything load-bearing belongs in the body rather than behind a reference pointer, that reference files should be small enough to be cheap to open — splitting a 3,000-token auth.md into a 400-token summary of the current flow and a longer appendix changes the odds considerably — and that the audit has to include at least one trace from a long session, because the fresh-session trace is the flattering one.

Answers that carry their provenance#

The engineer who lost two days had no way to interrogate what he was told. The answer named a flow and showed a code sample, and the file it came from, its age, and the fact that a current version existed twenty metres away in the same repository were all invisible. A line in the skill body fixes most of that:

When you answer from a file in reference/, name the file and its `copied` date
in your reply, in this form:

    Source: reference/auth.md (copied 2025-11-14 from docs/platform/auth.md)

If the file is more than 90 days old and the repository contains the source
path, read the source instead and say which you used.

The second rule is the one that would have prevented this particular incident, and it works because the agent had filesystem access all along. Nobody had told it that the bundled copy was a copy of something reachable. The first rule is the cheaper win: once dates appear in answers, users start noticing them, and a reader who sees "copied 2025-11-14" against a question about an API that changed in February has the information needed to check. Provenance in the output turns a class of silent failure into something the person on the receiving end can catch, which matters because they are usually the only one in the loop with a reason to care.

What freshness does not buy you#

A current reference is not a correct one. The file can match its source exactly and still be wrong for the request in front of it, because the source is aspirational, or because it documents the flow for external partners when the caller is an internal service. Provenance checks answer "is this the same as upstream", which is a narrower question than "is this right", and the second question still needs a human who knows the system.

Freshness also does nothing about volume. Every reference file that loads spends context, and the run traced above spent 2,610 tokens on rate limits nobody asked about, purely because one sentence in SKILL.md gestured at a directory. Naming files individually against the situations that need them — "for token exchange, read reference/auth.md; for retry and quota behaviour, read reference/rate-limits.md" — is the same discipline the description needed in the second chapter, applied one level down.

Neither of those is the problem the trace manifest actually raised, though. That manifest listed the files from one skill and showed db-migrations sitting there unmatched, which is the easy case. The interesting case is the run where two skills both match, both bodies land in context, and one of them says never to backfill in the same statement as a schema change while the other says the repair has to be atomic. The trace will show you both files loading. It will not tell you which one the agent believed.