Near-miss passages catch evidence attached to a claim it never entails

A citation test suite is a set of cases, each pairing a question with the passages a generator will see, plus an assertion about what the resulting citations should be. Its purpose is to fail when the system attaches evidence to a claim that evidence does not establish. Whether it can do that is a property of the passages in it, not of the assertions written underneath, and that is the part most suites get wrong.

The property to aim for has a name. An evaluation has discrimination when its cases can be failed by a system lacking the capability being measured and passed by one having it. Without discrimination a suite still produces a number, still moves when the system changes, and still gets quoted in a review — it simply reports on something other than what its name says. The specific confusion this chapter is about is between two things that look identical at the level of a score: whether a system selects the passage on the right subject, and whether the passage it cites makes the sentence true.

The discrimination a test corpus has to provide#

Take a policy document that describes log retention per region. One passage reads Request logs in the us-east-1 region are retained for 30 days from the time of ingestion. Another, four hundred tokens later, says request logs for customers provisioned in eu-central-1 are retained for 7 days. A question arrives about retention for EU customers, the answer says 30 days, and the citation points at the first passage — a real span, in the right document, in the current version, quoted character for character. The span exists, the quote matches, and the sentence hanging from it is wrong.

Now ask what a suite would have to contain to catch that. Attaching the retrieved chunk with the highest embedding similarity to the claim, with no notion of entailment anywhere in the system, scores 0.97 on a corpus of obviously-right and obviously-wrong passages. The strategy that scores 0.97 is the same strategy that produced the retention answer, so a high score and a live defect are perfectly consistent; what separates the two situations is nothing about the system and everything about whether the corpus ever contained a case where the highest-similarity passage was the wrong one. Distinguishing a passage about retention from a passage about invoicing is a task lexical overlap solves at ceiling. A suite built only from that comparison measures topical selection and reports it as support.

Fixing that means seeding the corpus with passages that a similarity function likes at least as much as the true passage, and that do not entail the claim. A distractor, throughout this chapter, is a passage put in the corpus for the system to wrongly select. The distractors that do useful work are the ones that are hard rather than the ones that are wrong, and the ordinary distractor — a chunk about billing sitting beside a question about retention — is only the second.

Those exist naturally in real documents, which is why the failure shows up in production: policy documents enumerate regions, rate cards enumerate tiers, changelogs enumerate versions, and every one of those enumerations is a set of near-identical passages differing in one attribute. The construction recipe follows directly from that observation. Take the passage that genuinely supports the claim, copy it, mutate exactly one attribute, and leave both in the corpus. Mutating exactly one is what makes the pair informative: everything the two passages share is a token a similarity function will count in both, so the only thing that can distinguish them is the attribute you changed, and the attribute you changed is the thing a support check has to be sensitive to.

Each row needs two variants to be useful, and the two are not two versions of one test. In the first, the true passage stays in the corpus alongside the distractor, and the assertion is that the system cites the true one — this is the case that separates support from resemblance, because both passages are available and only one of them makes the claim true. In the second, the true passage is removed and only the near miss remains, and the assertion is that the system declines to cite, or ideally declines to make the claim. The second variant is the one most suites omit, and its absence is exactly what leaves the worst behaviour untested: a generator required to produce evidence for every claim will produce the closest thing available, and the first variant can never observe that, because in the first variant the closest thing available happens to be correct.

Writing the cases so they test one thing#

A case that leaves retrieval free is testing retrieval and citation at once, and when it fails you will not know which. The reasoning is short and worth following, because it explains a piece of test design that otherwise reads as fussiness. A case's outcome depends on the passages in the context window; if retrieval chooses those passages, then a change to the retriever changes the outcome; so a failure is consistent with both a citation defect and a retrieval defect, and the suite cannot tell you which one you have. Pinning the retrieved set removes retrieval from the causal chain. The case specifies the chunks that will be in the context window, in order, and the generator is invoked against exactly those. Retrieval quality gets its own suite, and both suites become interpretable.

- id: retention-eu-scope-both-present
  question: "How long do we retain request logs for EU customers?"
  context:                      # exactly what the generator sees, in this order
    - chunk_id: policy/retention.md#tok:0
      text: >
        Request logs in the us-east-1 region are retained for 30 days from the
        time of ingestion, after which they are deleted from hot storage.
    - chunk_id: policy/retention.md#tok:400
      text: >
        Request logs for customers provisioned in eu-central-1 are retained for
        7 days from the time of ingestion, in line with the EU addendum.
  expect:
    answer_contains: "7 days"
    cited_chunks_include: ["policy/retention.md#tok:400"]
    cited_chunks_exclude: ["policy/retention.md#tok:0"]
    family: scope

- id: retention-eu-scope-true-absent
  question: "How long do we retain request logs for EU customers?"
  context:
    - chunk_id: policy/retention.md#tok:0
      text: >
        Request logs in the us-east-1 region are retained for 30 days from the
        time of ingestion, after which they are deleted from hot storage.
  expect:
    behaviour: decline        # no claim about EU retention, or a claim with no citation
    family: scope

Roughly sixty of these is enough to be informative and small enough to hand-check when the numbers move, and the figure is arithmetic rather than a rule of thumb. Seven families, at four or five cases each, gives twenty-eight to thirty-five cases; each needs both variants, so the count doubles to somewhere between fifty-six and seventy. Building them takes about a day if you mine the mutations from real documents rather than inventing them, and mining is the better source because the distractors that occur naturally in your corpus are the ones production will actually retrieve. An invented distractor tests whether the checker can be fooled in principle; a mined one tests whether it will be fooled by the documents you have.

Asserting entailment rather than overlap#

The assertion in cited_chunks_include catches the case where the wrong chunk was picked. It does not catch a citation into the right chunk that lands on the wrong sentence, and it does not generalise to production traffic where there is no labelled answer. For both you need a predicate over the pair (span, claim) that returns whether the first establishes the second.

Lexical overlap is the tempting implementation and it is measurably the wrong one on exactly these cases. Scoring the retention example by token F1 between claim and span — a single number combining how many of the claim's words appear in the span with how many of the span's words appear in the claim — gives the distractor 0.71 and the true passage 0.62, because the distractor shares the longer noun phrase. The ordering is inverted, by a gap of about nine hundredths, and the inversion is not bad luck. Every family in the table above is constructed so that the distractor wins or ties on surface similarity, since that is what the recipe does: it makes the distractor a copy of the true passage with one attribute changed, which leaves it sharing more tokens with anything phrased like the true passage than the true passage shares with a differently-phrased claim. A near miss is precisely a passage that surface similarity ranks first.

Natural language inference, or NLI, gives you the predicate directly. It is the long-standing task of taking two pieces of text and labelling the relation between them: premise is the cited span, hypothesis is the claim, and the three available labels are entailment, where the premise makes the hypothesis true, contradiction, where it makes it false, and neutral, where it settles nothing either way. The label you require is entailment. A cross-encoder fine-tuned on MNLI — a model that reads premise and hypothesis together in one pass and emits one of those three labels, trained on a public corpus of several hundred thousand hand-labelled pairs — runs at roughly 15 ms per pair on a CPU and costs nothing per call, which makes it cheap enough to run over every citation. An LLM judge with an explicit rubric — a general-purpose language model prompted to return the same three-way verdict, with the criteria written out for it — costs about $0.0004 and 120 ms per pair, and handles the multi-sentence and cross-passage cases the small model is poor at. Running both and treating disagreement as a review queue is a reasonable production posture, and it also tells you where the small model can be trusted alone.

Two details make the difference between an entailment check that works and one that agrees with everything, and the first is the stumble in this chapter because it looks like the opposite of what it is. Clipping the premise tightly to the cited span feels stricter — less text, so less room for the checker to find something that fits. It is looser, and the retention case shows why in one step. A span clipped to retained for 30 days no longer contains the word us-east-1, and the region was the only thing making the passage the wrong evidence. Feed that clipped premise to a checker and it will happily entail a claim about the EU, because as far as the checker can see there is nothing regional in the premise to conflict with. The defect being hunted has been removed from the input to the tool hunting it. Feed the premise as the full sentence containing the span, with the span's own boundaries recorded separately, and the scope-bearing words come back. The second detail is that the hypothesis must be a single proposition, which is what the claim-splitting rule from chapter 2 already produces; a two-clause hypothesis makes an entailment model return a verdict on the clause it found easier.

Report per family rather than in aggregate. A single accuracy number is dominated by whichever family has the most cases, and the whole point of the construction is that the families fail differently.

That table is read across, not down, and what it says is in the gaps between adjacent columns rather than in any level. Three different gap profiles appear in it, and each one names a different thing to change.

Scope, date, quantity and entity all make their largest jump between the middle column and the right one — between a bare-span premise and a containing-sentence premise. That is the same mechanism four times over. In each of those families the attribute distinguishing the true passage from the distractor is a scope-bearing word, and a bare span is a clip that can lose it. Widening the premise is not a better model; it is the same model given back the words the clip removed. It is also, usefully, a change to how you call the checker rather than a change of checker, so it is the cheapest improvement available in the whole table.

Polarity has the opposite profile. Its big jump is between the left column and the middle one, from a similarity score to an entailment model, and it barely moves afterwards. The reason is that a negation lives inside the span itself — is not included is not a scope-bearing word sitting outside the quoted fragment — so widening the premise adds nothing the checker did not already have. What polarity needed was a model that reads word order at all, which a bag of tokens and a mean-pooled embedding both fail at by construction, and once it has one the family is nearly solved.

Aggregation is flat across all three columns, and flatness is the strongest statement in the table. No configuration of premise or checker moves it, and the reason is that the target of the claim is not in any premise: "most" is a statement about a distribution over the whole corpus, and every candidate premise is a passage. A checker asked whether a passage entails a claim about a population it does not contain has no way to say yes truthfully, so the score stays near the floor whatever you feed it. What that licenses is a rule rather than a tool: the generator may not emit aggregate claims from prose evidence, which is a design decision made at generation time and not a checker that has yet to be built. That failure returns in the final chapter, where it is the whole subject.

The vocabulary for the two directions of this measurement is settled. Citation precision is the share of emitted citations whose span entails its claim; citation recall is the share of claims fully supported by the set of citations attached to them. The two are predicates over different populations — one iterates over citations, the other over claims — which is why a claim with no citation at all is a recall failure and cannot be a precision failure. The ALCE benchmark, in "Enabling Large Language Models to Generate Text with Citations", defines both carefully and is worth reading before you invent your own variants, if only because it makes explicit that a claim with no citation is a recall failure rather than something outside the measurement.

Truth, traffic and the checker's own error rate#

A near-miss suite does not tell you that the cited passage is true. It verifies a relation between a span and a claim, and a document containing a stale policy will produce citations that pass every check while asserting something the company stopped doing in March. Support and truth are different properties, and no arrangement of premises and hypotheses inside the suite reaches the second one.

It does not measure production either, and the gap between the two is worth stating in the terms the suite is actually built in. Sixty pinned-context cases run in about four minutes — a few seconds each, which is what pinning the context buys you — and they tell you whether a known class of defect is present in a build. They say nothing about the distribution of near misses in real traffic, and that distribution is what determines how often the defect fires. A build can carry a defect that never triggers because your documents happen not to enumerate anything, and a build can be clean on all seven families and fail constantly on an eighth you did not think of.

It also does not validate the checker. An NLI model reporting 0.91 on the scope family is itself a measurement with error, and treating it as ground truth means every subsequent disagreement gets resolved in its favour by default. The honest version of this work labels a couple of hundred pairs by hand and reports the checker's agreement with those labels per family, so that when the checker later disagrees with a reviewer you know which to believe — and per family rather than overall, for the same reason the results table is per family. The interactive explainer "Attribution and Citation: Sourcing LLM Outputs" walks through an NLI-based evaluation implementation in enough detail to make that calibration concrete.

A suite that passes gets you a system whose citations, on average, point at text that supports the claim. Averages are what a suite can offer, because it runs over cases somebody chose in advance. The person reading a particular answer needs something narrower than an average: they need to see, for this sentence, that this text says it. At the moment the only thing standing between them and that is whatever the interface does when they click.