Paired prompts and attribution

Chapter 3 left a diagnosis that was well supported and not proved: a pricing assistant quoting $20 while a chunk containing $24 sat at rank one in the assembled prompt. The natural experiment is to remove the chunk and see what changes, and the instinct is right — the pairing is cheap, it takes an afternoon, and it produces something a design review can look at.

The instinct that follows it is the one that fails. Having two answers side by side suggests that the parts of the with-context answer which also appear in the without-context answer are recalled, and the parts that do not are retrieved. That rule is a claim about attribution, and attribution is exactly the thing a single generated text cannot support. This chapter separates what the pairing licenses from what it does not, so that the finding survives a sceptical reviewer rather than collapsing under the first objection.

The two answers and the highlighter#

Here is a pair, from the same pinned revision, the same system prompt and the same user question, differing only in whether the retrieved chunk was included:

=== A: no context ===
The Team plan is $20 per seat per month, billed monthly. Annual
billing is discounted — usually around 20% — and the plan includes
standard email support with a next-business-day response target.
Seats can be added mid-cycle and are prorated.

=== B: with the current pricing chunk ===
The Team plan is $20 per seat per month, billed monthly, or $19.20
per seat with annual billing. It includes standard email support
with a next-business-day response target, and seats added mid-cycle
are prorated to the end of the billing period.

The chunk says $24, and says nothing about response targets. Answer B is more specific than A, contains a derived annual figure, and reads throughout as though it were drawn from the document. It also carries the wrong price and a support commitment that appears in neither the document nor the current policy.

The rule a reader reaches for on seeing two answers side by side is a highlighter: mark whatever appears in both as recalled, and whatever appears only in B as retrieved. It is tempting because it is how attribution genuinely works for a human writer with a source open beside them. That writer really does select a source per sentence, so a seam in the finished text corresponds to a seam in the process that produced it, and two colours recover something real about how it was written. The rule carries that structure over to a model, and the structure is what does not carry.

Apply the highlighter rule and it runs out of colours almost immediately. "Standard email support with a next-business-day response target" appears verbatim in both, which by that rule makes it recalled — and it is, which is why it is wrong, since the current policy dropped next-business-day targets for the Team plan a year ago. Meanwhile "prorated to the end of the billing period" is phrased differently in B than in A, which by the same rule makes it retrieved, and the document does say something about proration. So the rule got one right and one wrong, and there was no way to know which was which without checking both against the truth, at which point the pairing has stopped contributing anything.

Read those two answers as a diff and you can conclude, correctly, that the context did something: B has details A lacks, and the annual figure is arithmetic performed on a number in the room. Read them as an attribution and you are stuck. The $19.20 is an annual per-seat figure the model arrived at by applying a memorised discount rate to a monthly price, and neither that rate nor the figure it produced appears anywhere in the document — but the document does raise annual billing, so the sentence is not purely recalled either. Which monthly price the rate was applied to is not recoverable, because nothing in either source states the operation, and that is what makes a figure of this kind the hardest case for any scheme that tries to attribute one. The response target is in neither source, which the pairing cannot tell you, because you only find that out by checking. And the price is identical in both answers, which is the least informative outcome the experiment can produce.

What a paired prompt establishes#

Three things, and they are worth stating precisely because each one is genuinely useful.

It puts the model's parametric belief on record. Answer A, produced with no documents at all, is a direct readout of what the weights hold about your subject matter. That is a useful artefact in its own right: it tells you which of your facts the model has a strong prior about and will therefore fight you over, and which ones it has never seen, where supplied context will be used without contest. Chapter 3's table of context-wins rates per fact was built from exactly this measurement, and building it requires the no-context arm.

It measures whether the context changed the answer at all, in aggregate. Run the pair across a suite rather than once and the rate becomes the finding. And it localises: a system in which A and B are indistinguishable across the board has a prompt-assembly bug or is dropping the context entirely, and you find that out in an afternoon rather than after three weeks of tuning a retriever. The pairing is a cheap, coarse instrument, and coarse instruments used early save a lot of time.

The strength of that last inference comes from the suite rather than from any one pair. A single identical pair is weak evidence, because the two sources can simply agree with each other. Across a suite it is strong, because a document that is being read contributes to every token the model produces, so it leaves marks that have nothing to do with the fact under test — an extra detail, a different ordering, a phrase picked up from the page. A whole suite in which nothing moves anywhere is much harder to explain by agreement than by the context never arriving.

Building the suite is the part that takes the afternoon, and the useful discipline is to write down, per question, the value the document supplies and the value you expect the model to recall, before running anything. That pre-registration is what makes the interesting cases visible: without it, "the answers were the same" is compatible with the two sources agreeing, and you cannot separate the uninformative agreements from the interesting ones after the fact without re-reading every pair by hand. It also forces you to notice which of your facts you have no prediction for, which is itself worth knowing — a fact where you cannot say what the model would recall is a fact you have not measured.

PAIRS = load_questions("pricing_and_policy.jsonl")  # question + expected value

def paired(question: str, chunk: str) -> dict:
    without = call_model(**FIXED, user=question)
    with_ctx = call_model(**FIXED, user=f"{chunk}\n\n---\n\n{question}")
    return {"question": question, "without": without, "with": with_ctx}


def classify(row: dict, supplied_value: str, recalled_value: str) -> str:
    a, b = row["without"], row["with"]
    if supplied_value in b and recalled_value not in b:
        return "context_used"
    if recalled_value in b and supplied_value not in b:
        return "memory_survived"
    if supplied_value in b and recalled_value in b:
        return "both_present"        # the model hedged or contradicted itself
    return "neither"                 # a third value appeared; investigate

That classifier works — for the one fact you named in advance. It works because you told it the two candidate strings to look for. Every other claim in the answer goes unclassified, and the interesting failures live there.

Why there is no seam#

Attribution fails here for a reason that sits below wording, and it is the same machinery chapter 2 used to explain why there is no refusal. Producing an answer is one loop: the model computes a distribution over the tokens that could come next, one of them is chosen, and the loop runs again with that token appended. Every pass of that loop is conditioned on everything in front of it at once — the system prompt, the supplied chunk, the question, and whatever the model has already written — with the weights entering the same computation that the context enters. There is no earlier stage in which a claim is composed and no later stage in which it is rendered from a chosen source.

What follows from that is stronger than the model being unwilling to say where a claim came from. Nothing in that loop ever holds "the Team plan is $20" as an object, so there is nothing for a provenance field to be attached to, and the quantity an attribution report would be a report of is never formed anywhere in the computation. It is not lost somewhere between the model and the output, and it is not being withheld. A seam in the text would have to correspond to a seam in the process, and the process has none. That is also why no prompt wording recovers attribution and why no amount of careful reading recovers it either: reading more carefully is a way of finding information that is present in a text, and this information was never present.

The model does not assemble an answer by selecting a source per claim and then rendering it. It produces a single continuation conditioned on everything in front of it, with the weights and the context contributing to every token. The practical consequences show up in three forms.

The first is coincidence. Most claims in a well-functioning system are supported by both sources — the model already knew that seats are prorated, and the document says so too. Agreement carries no attribution information at all, and it is the majority case, so most of the answer is silent about where it came from no matter how carefully you read.

That majority is not an accident of this example, at least where the supplied documents describe something public. A pricing page is documentation about a product the world has also written about, and a model that has read the world has read a good deal of that writing, so the page and the weights agree on nearly everything except the handful of values that recently moved. The better the retrieval system works, the larger the agreeing majority becomes, and the smaller the fraction of the answer on which a pairing has anything at all to say.

The second is paraphrase. Where the model states a fact that is in the document, it usually states it in the phrasing it learned rather than the document's phrasing, because fluent restatement is what the model was trained to do. Two documents describing the same policy in different words produce nearly identical answers, so lexical overlap with the source is a weak signal for use and its absence is not evidence of non-use.

The reason is the same loop again. Each word of the answer is whatever the model computes as a likely continuation, and what it has learned to produce is the register of the subject rather than a copy of whatever is sitting in front of it. Copying happens where the document's phrasing is also the likeliest continuation, which is common for a rare identifier and uncommon for an ordinary sentence about billing. Overlap therefore tells you that two sources happened to agree on wording, which is a fact about the wording rather than about which of them was used.

The third is genuine merge, and the $19.20 is a clean specimen. The document raised the topic of annual billing; the discount rate came from memory; the base price came from memory; the arithmetic happened at generation time. That number did not come from a source — it came from a computation over both — so any classification scheme with the categories "from context" and "from memory" has no cell for it.

Follow that figure back and there is nowhere it could have been copied from. Answer A does not contain it, so it is not something the model volunteers about the Team plan unprompted. The chunk does not contain it either, which is what it means for the arithmetic to have happened at generation time, even though the chunk is what put annual billing on the table as a subject. A search of either source for the string therefore comes back empty, and a classifier asking which of two sources a claim came from has nothing it can return, because this claim came from neither and from both — a memorised discount rate and a monthly price on one side, the document's mention of annual billing on the other. Every derived figure in an answer has that property — a total, a proration, a per-unit rate — and derived figures cluster in exactly the answers people trust most, because arithmetic reads as diligence.

What defeats an attribution scheme there — including the citation mechanism later in this chapter — is that a derived figure gives it nothing to point at. A claim that was copied has a span containing it, so a correct citation exists somewhere to be found. A number that was computed has no such span by construction, because the computation happened while the answer was being written and left no text behind it. Asked to cite one, a mechanism can only point at a span about the same subject — the sentence that raised annual billing, say — and that is the near-miss shape which reads as relevant to a reviewer while supporting nothing.

Laying the four possible outcomes out makes the limits of the instrument explicit:

Row three is the one everyone reaches for, and it is a hint rather than a proof. A reviewer can accept the diagnosis in row three and still reasonably ask whether the chunk actually reached the model in a readable form — whether it was truncated, mangled by the template, or placed somewhere the model treated as boilerplate. You have not eliminated those, because the only thing you changed was the presence of a document whose content the model already agrees with in every respect except the number under test.

Citations do not close the gap#

The natural next idea is to require the model to cite. Ask for a quoted span from the supplied document beside each factual claim, and attribution becomes a rendering problem rather than an inference problem.

It helps and it does not settle it, for a reason that should be familiar by now: attaching a citation is itself a learned text behaviour, and the model can perform it for a claim it did not take from the cited text. In a run of sixty claims produced under a cite-your-source instruction, fifty-one carried spans that genuinely contained the claim, nine carried spans that did not — three of those quoting a nearby sentence about the same topic, and six quoting text that merely mentioned the same entity. The failure rate on citation is lower than the failure rate on the underlying facts, which makes it a useful mitigation, and it is not zero, which makes it a poor instrument for proving a diagnosis.

Why it cannot be an instrument follows from the section above. A citation is text, produced by the same loop that produced the claim beside it and conditioned on the same things, so asking for one does not open a channel into the computation. It adds an output. The quoted span then has to be checked against the document exactly as the claim has to be checked against the truth, and a mechanism whose output requires checking is a mitigation rather than a proof. What a citation genuinely buys is that the checking becomes cheap — a span to look at instead of a fact to research — and that is worth having on its own terms.

The nine failures also cluster in a way that makes them worse than a random nine percent. They concentrate on claims where the document is nearly on topic — a span about annual billing cited for a claim about monthly billing, a span about the Enterprise tier cited for a Team tier limit. Those are precisely the citations a reviewer skims and accepts, because the quoted text looks relevant. A citation mechanism whose errors are concentrated on plausible-looking near misses gives a reviewer more confidence than they had before while leaving the residual risk in the hardest cases to catch.

That concentration is not bad luck. A span is selected the way every other stretch of output is selected, by being a likely continuation given the claim and the document in front of it, and being nearly on topic is what makes a span likely. So the spans that get cited wrongly are the ones that read as though they belong, and reading as though it belongs is the whole of what a reviewer checks when skimming a citation. The property that gets a wrong citation produced and the property that gets it accepted are the same property.

There is real work on making this rigorous rather than heuristic. Provider-side citation features that bind claims to character offsets in the supplied document are stronger than asking the model to quote, because the binding is produced by the serving stack rather than by generation. Research approaches like Seeing through the Conflict: Transparent Knowledge Conflict Handling in RAG go further and surface the conflict itself — flagging that the context and the model's own belief disagree, rather than silently resolving it. Those are the right long-term direction for a production system. Neither is something you can stand up this afternoon to settle an argument about whether to buy a reranker.

The pairing, then, is a good first instrument held to its actual resolution. It is not a claim-level classifier, and treating its output as one produces a confident, wrong report. It is not a substitute for checking the facts, since the third row of the outcome table only becomes visible once you know what the document says and what the truth is. And it is not a control for the possibility that the document was unreadable in the assembled prompt, because both arms leave that untested.

The gap in all three cases is the same. The document you supplied contains facts the model might already hold, so any agreement between the answer and the document is uninformative, and any disagreement has several explanations. What would fix that is a document containing something the weights cannot possibly contain — a value with no prior, where use of the context is unmistakable and non-use is equally unmistakable, and where a single re-run at identical settings settles the question. That is the manipulation the next chapter is built around.