A citation is checkable only when it opens the supporting passage

Atlas showed three citations under every answer, and in a quarterly survey of 84 claims handlers, 61% disagreed with the statement I trust Atlas's answers enough to act on them without checking elsewhere. The product manager's proposal was to raise the retrieval budget from three chunks to six, show all six as citations, and print the similarity score next to each so handlers could see how well supported an answer was. The instrumentation said something different about where the problem was.

Over 30 days and 22,400 answers, a citation was clicked on 6.2% of answers. Of those clicks, 71% opened a PDF viewer at page one of a document averaging 42 pages. The median session in the viewer lasted 38 seconds, and 26% of returns happened with no scroll event recorded at all, which is what giving up looks like in a log. Handlers were not distrusting the citations because there were too few of them. They had learned, over roughly a week of employment, that following one cost more than answering the question from their own knowledge of the manual, so they stopped following them, and an unfollowed citation provides exactly as much assurance as no citation.

The verification budget is a real budget#

Every check a user might perform sits against the cost of not performing it. Reading a two-sentence answer takes about 8 seconds. Re-deriving the answer from the manual by hand — opening the right document, finding the clause, reading around it — took handlers a median of 94 seconds in a small timed exercise. A citation is worth following only if it lands somewhere between those numbers, and comfortably below the second one; when following the citation costs 60 seconds because the reader has to search a 42-page document for a clause whose wording they do not know, the rational move is to skip it.

Anchor granularity is what sets that cost, and it moves the number by an order of magnitude.

The last row is the target, and the reason it needs with context is that a highlighted sentence with nothing around it is unfalsifiable — the reader cannot tell whether the next sentence says "except in cases of subsidence".

What a citation record has to carry#

Getting to a five-second check means the citation is not a document ID. It is a location plus enough redundancy to survive the document changing underneath it.

{
  "citation_id": "cit_4a81",
  "sentence_id": "s1",
  "doc_id": "hchm",
  "doc_title": "Household Claims Handling Manual",
  "doc_version": 7,
  "locator": { "page": 14, "section": "14.2", "char_start": 18422, "char_end": 18611 },
  "quote": "An appeal against a declined subsidence claim must be received within 60 days of the date of the decision letter.",
  "quote_sha256": "b1f0…c93a",
  "retrieved_at": "2026-03-04T09:12:44Z",
  "acl_scope": "claims_handler"
}

The quote and its hash are what make the record self-checking. Character offsets into a document are the fastest way to open the right place and the first thing to break, because a document re-exported from the source system shifts every offset after the point of the edit. Holding the quote lets the resolver fall back to a text search when the offsets no longer match, and holding the hash lets it distinguish the offsets moved from the passage itself changed, which are different events with different consequences for the reader.

Resolution therefore happens at click time rather than at answer time, and it has branches.

Branch E is the common case in a live corpus and the one most implementations get wrong by not having it — they compare versions, find a mismatch, and either open page one or say nothing at all. Branch H is rare and matters disproportionately, because a citation that cannot be resolved to anything is the strongest available signal that an answer should not be acted on, and it is knowable at click time with no model involved.

Access control is the branch that is missing from the diagram because it belongs earlier. On Atlas, an audit found that 9% of citations pointed at documents the requesting handler could not open, which produced both a dead end for the user and a small information leak in the document title. Filtering retrieval by the requester's ACL — the access control list attached to a document, which is the set of roles and users permitted to read it — rather than filtering the citation list after generation is the only version of this that works, since a passage the user may not read should not have shaped the answer either.

What the shallow fix does#

Adding citations is appealing because it is a change you can ship in a sprint and demonstrate in a screenshot. Atlas ran the six-chunk variant as an experiment for three weeks. Mean citations per answer went from 3.0 to 6.4. Follow-through fell from 6.2% to 3.1%, because a row of six markers reads as boilerplate in a way that three does not, and the survey question about trust moved by less than the confidence interval. The change also raised the rate at which the marked citation was not the passage supporting the adjacent claim, since the extra chunks were lower-ranked and the model attached markers to them opportunistically.

Surfacing retrieval scores fails for a more basic reason: a cosine similarity of 0.83 is not a probability that the answer is correct, it is not comparable across queries, and there is no action a claims handler can take differently at 0.83 than at 0.79. Where a score genuinely does carry decision value, the honest form is the coarse label from chapter 1, which at least states what it claims. Handing the user a raw internal number transfers the interpretation problem to the person least equipped to solve it, and it creates a false impression that the system knows how sure it is.

The four measurements that tell you where the problem is#

When a stakeholder says users do not trust the output, the productive response is to find out which of four distinct failures is producing the complaint, because they have different fixes and only one of them is about citations at all.

Follow-through rate — the share of answers where a citation is clicked — tells you whether the citation layer is being used. At 6.2% either the answers are trusted enough not to need checking or the checks are too expensive; the two look identical in this metric and are separated by the next one.

Verification success rate — of the clicks, the share where the reader reached the supporting passage — tells you whether the anchor is doing its job. Atlas measured this by logging whether the viewer's highlight was in the viewport when the user left, which is imperfect and was enough: 34% before anchoring, 88% after.

Post-check correction rate — the share of verifications after which the user changed what they did, by editing the drafted letter, re-asking, or escalating — is the one that says whether checking is worth anything. A rate near zero means the checks are ceremony; the Atlas number was 11%, which is high enough that the check was doing real work and low enough that making every answer require a check would have been a poor trade.

Time to verify, measured from click to return, is the cost side of the budget. It is the number that moved most on anchoring, from a median of 38 seconds to 6.

None of these is citations per answer, and a team that reports citation count as a trust metric has picked the one quantity that can be improved without changing anything a user experiences.

Where anchored citations still leave you exposed#

A perfectly anchored citation says a passage exists and where it is. It does not say the passage supports the claim beside it, and the gap between those two is wide enough to drive an incident through. During the anchoring work the team found answers where citation [1] opened clause 14.2 correctly, the clause did establish a 60-day window, and the answer's second sentence — about what the appeal must contain — was supported by nothing in that clause. The marker sat at the end of the paragraph and appeared, to any reasonable reader, to cover all of it.

Anchoring also does nothing about the corpus being wrong. Several of the distrust reports traced back to answers that were correctly retrieved, correctly cited and describing a procedure that had been superseded by an endorsement circulated by email and never added to the document store. The citation resolved, the passage was current in the store, and the answer was wrong. No amount of interface work fixes that; what the interface can do is make the document's last-revised date visible in layer 1, which turns an invisible failure into one a handler with local knowledge can catch.

Finally, citations are a poor fit for answers that are genuinely syntheses. When the model combines four passages to answer a question none of them answers directly, per-claim attribution is doing something more like a proof sketch than a reference, and the honest presentation is closer to showing the four inputs and the step that combined them than to sprinkling markers through the prose. Products that require this are usually better served by the layer 2 audit view of chapter 2 than by more markers in layer 0.

That leaves the specific problem of which words a resolved passage stands behind. Marking that at the level of the paragraph is what allowed the unsupported second sentence to hide inside a well-cited answer, and doing it at the level of the span forces a decision the paragraph-level version let everyone avoid: what to do with the sentence that has no support at all.