Hallucination

Read these first

• Autoregressive generation

What you will know

• Distinguish a hallucination that contradicts supplied context from one that invents beyond it

  Label each claim against the provided passage; one response can contain both a contradiction and an unsupported addition.

• Compare the same prompt with and without retrieved evidence, then identify which hallucination type retrieval reduces

  Additional evidence mainly constrains unsupported additions, while a model can still misread or contradict text already in its context.

• Trace why a plausible false answer follows from next-token prediction

  Likelihood rewards continuations that fit learned patterns, not a separately verified truth value.

• Detect both kinds in a real output

  Use an annotated answer and source passage to mark the exact span that conflicts with evidence or lacks support.

• Break a “do not hallucinate” instruction with an ambiguous or absent fact and measure the result

  A refusal or confident disclaimer may change the wording without improving factual accuracy, especially when the prompt supplies no verifiable answer.

Chapters

1. 1

   Hallucination and next-token prediction

   A hallucination is a claim a language model states as fact when nothing supports it, delivered in the same fluent register as the claims it gets right. This chapter defines the term, traces the failure to the training objective — which scores how well a continuation fits learned patterns and never consults a truth value — shows the token distribution at the moment a model commits to a fabricated citation, and explains why lowering the temperature, beam search and calibration all leave the problem untouched.

   Read chapter
2. 2

   Intrinsic and extrinsic hallucination

   A supplied policy states a 14-day refund window and the model answers 30 days: that is an intrinsic hallucination, a claim contradicting content that was in the prompt. A claim with no counterpart in the supplied material at all is extrinsic. This chapter defines both against the material actually placed in the prompt, crosses them with truth in the world to give the four cases that stop labelling arguments, and shows why one response routinely contains both kinds interleaved with claims that are perfectly fine.

   Read chapter
3. 3

   Claim-level annotation

   A response-level 'hallucinated' checkbox loses the type, the density and the location of every failure it records. This chapter sets out the four-step procedure that replaces it — decompose the answer into atomic claims, align each to a span of the supplied context, label it supported, contradicted or unsupported, and mark claims inherited from an earlier error — along with the storage format, the scoring code, how to draw the sample, and what the pass costs in annotator time and inter-annotator agreement.

   Read chapter
4. 4

   Retrieval-augmented generation

   Supplying evidence to a model means running a search over a chunked document collection and pasting the top-ranked passages into the prompt as ordinary text. This chapter walks the mechanism end to end — chunk sizes and their boundaries, the embedding index, the ranked list with its similarity scores, and the string assembly that hands it all to the model — and then draws the stage boundary precisely, so that 'the correct document came back ranked first' can be read for exactly what it exonerates and what it leaves open.

   Read chapter
5. 5

   Measuring the effect of retrieval

   Run one prompt with retrieved evidence and without it, label both arms against the same gold passages, and the unsupported-claim rate falls by 85% while contradictions do not move at all. This chapter sets out the experiment design that makes the two arms comparable, reports the paired result with bootstrap intervals, explains why extra material removes the need to invent without compelling the model to read, and sweeps k to show where more evidence starts making the total worse.

   Read chapter
6. 6

   Anti-hallucination prompt instructions

   Adding 'Do not hallucinate. Only state facts you are certain of.' to a system prompt raises correct declines on unanswerable questions from 12% to 61%, raises wrong refusals on answerable ones from 1.4% to 9.5%, and leaves accuracy-when-committing exactly where it was. This chapter measures five phrasings against a three-way question split, prices the over-refusal in agent-hours, shows the hedged-and-wrong output the instruction creates, and explains why an instruction can only reach the register.

   Read chapter
7. 7

   Verifying claims against cited spans

   Require the answer to emit, for each claim, the passage and the verbatim sentence it rests on; then run three checks over those pairs before the response is returned — the quote locates, the span is on topic, a natural-language-inference model scores entailment. This chapter builds that pipeline with its schema, its string-matching fallback, its thresholds and their measured precision and recall, prices the repair loop at 18-24% extra per request, and sets out what a faithfulness check structurally cannot catch.

   Read chapter
8. 8

   Grounding commitments and measured rates

   Legal asks for a written promise that an assistant will never state anything absent from the source documents. No prompt makes that true, so what can be signed is different: a deterministic gate over the claims that carry a checkable handle, a verifier-plus-abstention path for the rest, and a per-claim rate measured on a stratified sample with a confidence interval, a threshold, a cadence and a defined breach response. This chapter writes that document, sizes the sample, and names the domains where a rate is not an acceptable answer at all.

   Read chapter