Showing a faithfulness judge the gold answer stops it measuring grounding

A pipeline that searches a document collection for passages relevant to a question, pastes those passages into a prompt, and asks a language model to answer from them has exactly one number that says whether the model stayed inside the evidence it was handed. That number is faithfulness: the share of the claims in the generated answer that the retrieved passages entail. Grounding is the property it measures, the property of every assertion in an answer having a passage behind it. Faithfulness is the only metric in a standard evaluation suite that measures grounding at all, which is why a faithfulness score that is wrong in the flattering direction is worse than having no faithfulness score: there is nothing else in the suite positioned to catch it.

The score comes out of a judge — a second language model, shown some subset of the artifacts an evaluation sample carries, and asked to return a verdict. A sample carries four artifacts. There is the question a user asked, the answer the system generated, the retrieved passages the generator was given, and the ground truth, also called the gold answer, which is a reference answer written by a human who knew what the right answer was. Which of those four the judge is shown is not a detail of harness plumbing. It decides which quantity the returned number is, and the name on the dashboard does not change when the quantity underneath it does.

One sample type for every metric#

Evaluation harnesses converge on a shape, and the shape is a single sample type with a uniform interface, so that adding a metric means adding one entry to a dictionary rather than threading an argument through a runner.

@dataclass(frozen=True)
class EvalSample:
    question: str
    answer: str
    contexts: list[str]
    ground_truth: str          # the human-written reference answer

Metric = Callable[[EvalSample, Judge], float]

def run(samples: list[EvalSample], metrics: dict[str, Metric], judge: Judge):
    return {name: mean(m(s, judge) for s in samples) for name, m in metrics.items()}

That code is good by every ordinary standard, and the step that makes it dangerous is easy to walk past. A uniform interface has one argument list. One argument list covering every metric has to be the union of what all the metrics need. So every metric now receives every artifact, including the artifacts it was never defined over, and the convenience at the call site is paid for downstream by each judge seeing more than its own question requires.

On the harness above, that change produced a measurable move. The next nightly run reported faithfulness at 0.94, up from 0.78, on a pipeline whose retriever, prompt, model and corpus had not changed. Nobody had touched generation in three weeks. The dashboard had been redrawn to say that the system's grounding problem was largely solved, and the only thing that had actually happened was that the faithfulness judge started receiving ground_truth in its prompt along with everything else.

A claim that is correct passes as a claim that is supported#

Faithfulness asks whether the retrieved passages entail each claim. A judge holding the reference answer is answering a different question, because the reference answer entails a great many claims the passages do not, and a language model reading a prompt that contains supporting text will find that support wherever it sits.

The judge is not malfunctioning, and it is worth being exact about why, because the instinct to blame the model sends the repair to the wrong place. A judge is a language model, and a language model conditions on the whole of its prompt rather than on a designated region of it. The prompt now contains a sentence that entails the claim. The judge is asked whether the claim is supported by the material it has been shown, and the reference answer is material it has been shown. A verdict of supported is therefore correct with respect to the prompt the judge received, and wrong only with respect to the question the metric's name promises. Nothing in that chain is a model failing at its task; the failure is upstream, in what got put in front of it.

The verdict record makes the substitution visible:

{
  "claim": "Audit log exports are delivered daily at 02:00 UTC.",
  "verdict": "supported",
  "evidence": "Exports are delivered once per day at 02:00 UTC."
}

That evidence string is not in contexts. It is a sentence from ground_truth, written by a support engineer who knew the real cadence. The retrieved passages say nothing about when exports run, which is precisely the fact the pipeline should be penalised for: the generator produced a correct cadence from its own weights or from a lucky guess, with no passage behind it, and the metric whose entire job is to notice that marked it supported.

The effect is largest exactly where you can least afford it. Claims the model gets right from parametric knowledge — the facts it absorbed into its weights during pretraining, with no retrieved passage behind them — are the ones most likely to appear in a reference answer, so the leak converts every well-known fact from an unsupported claim into a supported one, while leaving the genuinely wrong claims flagged. That is why the number moved sixteen points rather than two, and why it moved in the flattering direction. A team looking at 0.94 concludes the generator stays inside its evidence, and the pipeline is one corpus change away from repeating a superseded cadence with a citation attached.

The direction of that error is forced rather than observed, and the argument is short. Adding the reference to the prompt only ever adds text. A claim that some passage already entailed still has that passage in front of the judge, so it stays supported. A claim no passage entailed may now be entailed by the added text, so it can turn from unsupported to supported. No claim can move the other way, since nothing was removed. Claim by claim, then, the leaked score is greater than or equal to the correctly wired one, and a leak of this shape can only ever flatter. A metric with a one-sided error is a metric that will never raise an alarm about itself.

Telling the judge not to use the reference does not repair this. On the same 400 samples, adding "Judge support using the CONTEXT section only. Ignore the REFERENCE ANSWER section." to the verify prompt moved the score from 0.94 to 0.91, against a correctly wired baseline of 0.78. Thirteen points of the leak survived an explicit instruction, because attention does not partition cleanly on a heading and a claim that reads as true in the surrounding text gets marked true. The reliable fix is not to put the string in the prompt.

That instruction is worth naming as a stumble rather than filing as a disappointment, because it is the defence most teams reach for first and it is tempting for a good reason: it costs one line and no rearchitecting. What makes it a bad defence is that its strength does not vary with the thing it is supposed to police. The instruction reads the same, and the judge's prompt contains the same reference text, whether the passages supported a claim or said nothing about it. Three of the sixteen points is the measured size of what a wording change can buy, and thirteen is what it cannot.

Four readings of the same metric#

Four numbers have now come off one metric on one set of samples, and each is a different wiring rather than a different pipeline. With the contexts alone, faithfulness reads 0.78. With the contexts and the reference both present, 0.94. With both plus the instruction to ignore the reference, 0.91. And with the contexts blanked entirely and the reference still present, 0.71. Reading those four against each other says more than any one of them alone.

The first relationship is a partition, and it closes exactly. Three points separate the leaking run from the instructed run, thirteen separate the instructed run from the correctly wired one, and three plus thirteen is the sixteen points of total leak with nothing left over. The absence of a residual licenses less than it looks like it does: the three differences are consecutive gaps along one line, so they telescope and would sum exactly whatever the numbers were. What the partition earns is the split itself — how much of the gap a wording change reached and how much it did not — and no claim about there being no further cause.

The second relationship is a genuine agreement between two instruments. The 0.78 appears twice, from paths that share no machinery: the nightly score from the old harness, with its six separate call signatures, where the faithfulness function never had the reference in scope, and the new uniform harness run with the reference blanked out. Different call signatures, different code paths, one number. Because both runs used the same samples and the same judge, their agreement licenses a specific conclusion: the entire sixteen-point move is attributable to the presence of the reference string rather than to anything else the refactor changed. It licenses nothing about whether 0.78 is itself the right answer — the two readings share a judge, so a bias in that judge sits inside both and cancels out of the comparison.

The third relationship needs its condition stated first: it holds only if all four readings come from the same samples graded by the same judge, which is the case here. Treat the passages and the reference as two channels, each able to supply support for a claim. The passages alone leave a little over a fifth of claims unsupported, since that is what 0.78 means. The reference alone leaves a little under three tenths unsupported, which is what 0.71 means. A claim ends up unsupported in the leaking harness only if both channels fail it, and if the two failed independently the share failing both would be the product of those two shares — about six in a hundred, leaving a bit over ninety-four in a hundred supported. That is the 0.94 the leaking harness reported.

What that arithmetic licenses is a statement about the kind of thing the leak is. It behaves as a second, independent channel of support rather than as a softening or a rescaling of the first. A channel cannot be attenuated by asking the judge to disregard it; it has to be disconnected, which is what the three-point yield of the instruction and the thirteen-point survival already said from the other direction. Two independent arguments landing on one conclusion is the reason to act on it. What the arithmetic does not license is a claim that the two sets of claims are statistically independent in any deeper sense: at two decimal places a band of near-independent structures rounds to the same figure, so this is a consistency check rather than a proof.

The 0.71 is the chapter's real residual and it deserves a name. It is the share of an answer's claims that the reference alone entails, large because reference answers are written by people trying to be complete, and it is what a faithfulness judge reports when retrieval supplied nothing. That gives the sharpest statement of the failure available: with the reference in the prompt, the metric returns most claims supported in the world where the passages carried the evidence and in the world where there were no passages at all. A measurement that reads the same in both worlds cannot be evidence about which world you are in, and a grounding metric that cannot distinguish them is not measuring grounding under any reading of the word.

One figure underwrites all of the above. Repeating the same run five times put the judge's own variation at ±0.015, so the sixteen-point move is an order of magnitude larger than that band, and that ratio is the entire licence for reading these differences as signal rather than as run-to-run noise. Without a measured noise band, a sixteen-point gap and a one-point gap are the same kind of observation.

Each metric is defined over its own references#

Once you look at the four artifacts a sample carries and ask which of them each metric is defined over, the wiring is not a matter of taste. Three of the metrics are defined over two artifacts each, and they are three different pairs.

That table is derivable rather than conventional, and deriving it is what makes it possible to work out the right wiring for a metric that is not on it. Every row is a ratio. Faithfulness counts the claims in the answer that the contexts entail, over the number of claims in the answer; the top of that fraction ranges over the answer and the contexts, the bottom over the answer alone, and their union is the two artifacts in its Receives column. Answer relevance measures how much of the answer addresses the question, over the whole of the answer, so its union is the answer and the question. Context precision counts retrieved passages marked relevant, weighted by rank, over the passages retrieved. Context recall counts the supporting passages that were retrieved over the supporting passages that exist, and what fixes which passages exist is the reference. Answer correctness compares the answer against the human record of what is true.

So the Receives column is a requirement. It is not a policy anybody chose: it is the union of the sets the numerator and the denominator range over, and the ratio cannot be computed without those artifacts.

The Must-never-receive column is a different kind of statement, and conflating the two is what leads a team to treat both as negotiable. The definition alone says only that an artifact outside the union is unnecessary, which is not the same as harmful. What makes it harmful is one empirical property of this instrument: the judge is a language model, and a language model conditions on the whole prompt, so an unnecessary input is not an inert input. Must-never-receive is therefore a design decision about the harness, forced by the definition plus that one fact about judges. It would not be needed for a metric computed by a deterministic function, which can only read the fields it names — a string-overlap score can be handed the entire sample harmlessly, and a model judge cannot. Knowing which column is which is what tells you that the fix belongs in the harness rather than in the prompt.

There is a sharper way to say what an extra artifact does, and it explains why the failure is silent. An unnecessary input does not merely add noise; it makes a different ratio computable from the same prompt. Hand faithfulness the ground truth and the judge holds an answer and a human reference, which is exactly the pair answer correctness is defined over. Asked whether a claim is "supported", it can satisfy that word against either. The metric's name did not change and the quantity underneath it did. The substitution runs in the other direction too: hand answer relevance the contexts and the judge holds the pair context precision is built on.

Context recall is the row that stops this being a rule about never showing anything the gold answer. It is defined against the reference: the question it asks is whether the passages needed to produce the reference answer were retrieved, so the reference is its right-hand side in the same way the passages are faithfulness's. The mistake is not using the gold answer, it is using it in a metric whose comparison does not include it.

The question column has a similar shape. Faithfulness does not need the question — a claim is entailed by a passage or it is not, regardless of what prompted it — and including it drags the judge toward relevance, marking off-topic-but-supported claims down. Answer relevance must not see the contexts, because a judge that has read the passages will credit an answer for addressing the question when it addresses what the passages happened to cover.

The diagram is a projection onto those four sample artifacts, so context precision's per-passage relevance labels, which are not a field of the sample, do not appear as a source in it. Counting arrows in the two directions gives two different facts. Read forwards, into the metrics, every metric has exactly two arrows entering it — five metrics at two arrows each, ten arrows in all. Read backwards, out of the artifacts, they are not symmetric: the question and the gold answer feed two metrics each, while the generated answer and the retrieved contexts feed three each. Ten arrows over four artifacts cannot come out even, which is why the asymmetry is forced rather than accidental. At faithfulness, the two arrows in come from the generated answer and the retrieved contexts, and the absence of an arrow from gold answer is the whole content of this chapter.

An ablation finds the leak in one run#

You do not have to read the harness to know whether a metric is using an input it should not. An ablation removes one input and re-runs the same work to see whether the result depended on it, so run the metric twice over the same samples, once with the reference blanked, and compare.

blanked = [replace(s, ground_truth="") for s in samples]
print(run(samples, {"faithfulness": faithfulness}, judge))   # 0.94
print(run(blanked, {"faithfulness": faithfulness}, judge))   # 0.78

The reason that difference identifies the leak, rather than merely correlating with it, is worth setting out, because an ablation is only as good as what it held still. The two runs use the same samples in the same order, the same judge, and the same metric prompt. Exactly one field differs between them, and it differs in every sample. Anything else that could explain a change in the mean would have to take different values across the two runs, and by construction there is no such variable. The comparison is paired sample by sample, so the difference does not depend on the two runs drawing similar samples. The one condition it rests on is that the judge is repeatable enough for a difference to mean anything, which is what the ±0.015 over five repeats establishes.

A metric that is correctly wired is invariant to the value of every artifact it does not receive, so the two runs should agree to within judge noise, which on this set was ±0.015 over five repeats. Sixteen points of movement is a wiring bug, and the same ablation localises it without further reading: blank the question instead and faithfulness should not move either; blank the contexts and faithfulness should collapse toward zero, because a judge with no evidence should be unable to support anything.

Invariance turns wiring from something a reviewer asserts into something a test measures. A code review can report that a function does not appear to read a field. An ablation reports that the number does not change when the field does, which covers the cases a reviewer cannot see — a prompt template that interpolates the whole sample, a judge handed a serialised object, a helper two calls down that quietly reaches for more than it was given.

That last one is the sharpest of the three, and it is worth keeping in CI permanently. On the leaking harness, faithfulness with contexts=[] and the reference answer still present came back at 0.71 — a judge asked whether claims are entailed by nothing at all, reporting that most of them are. No amount of reasoning about prompt wording is as convincing as that number, and it takes one line to produce.

The mirror-image test applies to answer relevance. Blank the contexts and the score should not move; on a harness that hands the passages to the relevance judge it typically drops two to four points when you remove them, because the judge had been crediting answers for covering what the passages covered. The invariant is the same in both directions: whatever a metric does not compare against should be able to change arbitrarily without changing the score.

Making the wrong wiring fail to compile#

A convention documented in a comment survives about two quarters. Enforcing it in the type system costs a few lines and removes the failure mode, because a metric that cannot name a field cannot be influenced by it.

@dataclass(frozen=True)
class FaithfulnessInput:
    answer: str
    contexts: list[str]

@dataclass(frozen=True)
class RelevanceInput:
    answer: str
    question: str

@dataclass(frozen=True)
class RecallInput:
    contexts: list[str]
    ground_truth: str

METRICS: list[tuple[str, Callable[[EvalSample], object], Callable[..., float]]] = [
    ("faithfulness",     lambda s: FaithfulnessInput(s.answer, s.contexts),   faithfulness),
    ("answer_relevance", lambda s: RelevanceInput(s.answer, s.question),      answer_relevance),
    ("context_recall",   lambda s: RecallInput(s.contexts, s.ground_truth),   context_recall),
]

def run(samples, judge):
    return {name: mean(fn(project(s), judge) for s in samples)
            for name, project, fn in METRICS}

The projection functions are the interesting part. Each one is a lossy view of the sample, and the loss is deliberate: FaithfulnessInput has no ground_truth attribute, so a future change to the faithfulness prompt that tries to interpolate one raises an AttributeError in CI rather than silently improving the score. This is the same discipline as the six inspectable boundaries topic 9.2 built the pipeline around, applied to the eval harness instead of the pipeline — each stage gets exactly the inputs its contract names, and the artifacts it is not entitled to are not in scope.

The difference between the comment and the type is a difference in when the check runs and how often. A comment is checked by one reader, once, at review time, and the edit it needs to catch has not been written yet. A type is checked on every change by a machine that does not get tired, and the leak this chapter describes arrived as exactly that kind of later edit — a prompt template gaining a section, in a harness whose sample object was already carrying the string.

Two operational habits go with it. Record which judge model and which prompt hash produced each score, because a leak introduced by a prompt edit looks identical on a dashboard to a genuine regression, and the only way to separate them after the fact is to know what changed. And keep a small pinned set of samples with known-unsupported claims — a dozen answers where a correct fact appears in the reference and in no passage — as a canary: if faithfulness on those twelve rises above its recorded baseline, something has started leaking, regardless of what the aggregate does.

A wiring error, a weak judge and prompt injection are three different faults#

The leak in this chapter is a wiring error, so it is worth being clear about what it is not. It is not a judge quality problem — the same judge, given the correct two inputs, produced verdicts that agreed with human labels on the audit set. It is not prompt injection, where text somebody else controls smuggles instructions into a model's prompt, since nobody adversarial is involved and the offending string is one your own harness helpfully supplied. And it is not fixed by a stronger model, because a stronger judge is better at finding the support you showed it, which makes the inflation slightly worse rather than better.

That last one inverts the reflex that usually works, and it is the second belief worth naming. Upgrading the judge is the lever teams pull when a metric looks unreliable, and it is a reasonable lever for most metric problems. Here the quantity that decides the score is how reliably the judge locates entailing text inside its prompt, and the leak works by putting entailing text in the prompt. A better judge is better at precisely the step that produces the wrong answer. The usual defence does not merely fail to help; it is pointed the wrong way.

An analogy holds for part of this, and the limit is worth stating alongside the picture, because the limit is where the money is. The leaking harness is like a marker asked to check whether a student's working supports their answer, with the answer key stapled to the script — disclosed as an analogy, since no key is involved and no marking is going on. It breaks at the one place that matters. A human marker knows the key is not the student's working and can hold it separate; a language model's attention does not partition on a heading, which is what the three points the instruction bought, against the thirteen it did not, actually measure. Reaching for the analogy and stopping there is how a team concludes that a firmer instruction would do.

Where the gold answer does belong#

The rule is narrower than "keep the gold answer away from the eval", which is a conclusion people draw from this and then apply somewhere it does harm. Reference answers are how the labelled retrieval set gets built in the first place, since deciding which passages a question needed is a judgement about what the correct answer requires, and a team that decides gold data is contaminating ends up with no relevance labels and therefore no retrieval metrics at all. The reference belongs in metrics defined over it and in the labelling process that produces the ground truth; it does not belong in the prompt of a judge whose comparison is between an answer and the evidence that was retrieved.

The same distinction settles the case a reader is most likely to get wrong when extending this to a metric not in the table. The test is not "does this artifact look like an answer key". It is whether the artifact appears in the numerator or the denominator of the ratio the metric is defined as. If it does, withholding it makes the metric uncomputable. If it does not, supplying it to a model judge makes some other metric computable from the same prompt, and the name on the dashboard will not tell you which one you are reading.

A faithfulness score is only a statement about grounding while the judge that produced it has seen the answer and the retrieved passages and nothing else. Shown the reference as well, it reports something closer to correctness, wearing the name of grounding, and it reports it in the flattering direction because added text can only add support. The defence available at the prompt layer buys three points of the sixteen; the defence available at the harness layer buys all of it, because a projection that has no field for the reference cannot pass one. The thing to build is not a better instruction but a smaller argument list, and the thing to run in CI is the ablation that proves the argument list is what you think it is.

It also does not exhaust the ways a correctly wired faithfulness score can mislead you. The harness above hands contexts to the judge as a list, and every implementation of faithfulness in common use joins that list into one block of text before asking whether a claim is entailed. Support anywhere in that block counts as support. That is the right definition for the question faithfulness asks, and it means the score is indifferent to which passage did the supporting — which is fine until the answer starts putting a specific passage ID next to each sentence and users start clicking on it.