Debugging a single bad output

Read these first

• Error analysis
• Stop sequences, max tokens, truncation, silent output cutoff

What you will know

• Take one bad output and find the cause without guessing

  Record the rendered request, model settings, token counts, and a known-good trace so the conclusion can be reproduced rather than inferred from the visible text alone.

• Bisect a prompt to find the instruction responsible

  Instruction interactions can make the apparent culprit harmless in isolation, so verify the change with the same input and generation settings.

• Ablate context blocks one at a time to find the one that mattered

  A block may alter ordering, salience, or token budget even when its content is irrelevant, making ablation results sensitive to placement.

• Check whether the retrieved context even contained the answer

  A correct-looking retrieval score does not prove answer coverage; inspect the actual passages and confirm that truncation or formatting did not remove the evidence.

• Separate a systematic failure from a sampling fluke by re-running

  Temperature-zero repetition tests determinism, while multiple seeded samples reveal whether the observed output is merely low-probability behavior.

Chapters

1. 1

   Freezing a trace

   A trace is the complete record of one exchange with a model, and it stops existing the moment the system around it moves. Freezing one means capturing the request exactly as it left your service — rendered template, model id and resolved snapshot, sampling parameters and seed, per-block token counts, finish reason — plus a known-good trace of the same shape to diff against, so that whatever you conclude about a single bad answer can be reproduced rather than argued.

   Read chapter
2. 2

   Chat templates and the rendered request

   A messages array is a client-library convenience; what the model receives is one sequence of token ids produced by a chat template. When that template is skipped — an instruction-tuned model behind a completion endpoint fed \"System: ... User: ... Assistant:\" — there is no system prompt at all, only prose in one undifferentiated turn. Render the same messages through apply_chat_template, diff the ids against what was sent, and read the envelope fields before the prose.

   Read chapter
3. 3

   Answer coverage in retrieved passages

   Similarity ranks chunks against a query and says nothing about whether any of them states the fact the answer needed. Before blaming a model for inventing a refund window, read the retrieved passages and check coverage — then separate the three places evidence goes missing: never ingested, ingested but not retrieved, or retrieved and then removed by a token budget or mangled by formatting on the way into the prompt.

   Read chapter
4. 4

   Low-temperature replay and rate estimation

   Two questions hide inside \"can we reproduce it\", and they need two different replays. A pinned low-temperature run establishes that your own stack produces a sane answer from the recorded bytes; twenty to thirty runs at production sampling, with the seeds recorded, estimate how often the bad output appears. One duplicate refund in thirty is a systematic defect with a low probability, and the rate is also what decides whether a single edited re-run counts as evidence in the chapters that follow.

   Read chapter
5. 5

   Context ablation by substitution

   Removing a retrieved block and re-running changes three things at once — its content, the position of everything below it, and several hundred tokens of budget — so a correct answer afterwards has three candidate explanations and one run. Swapping the block for a same-length neutral passage holds layout and budget fixed and varies only content, which is what stands between a real finding and a corpus-wide filter that reinstates the bug the next time ranking shuffles.

   Read chapter
6. 6

   Instruction bisection

   Halving a 900-token system prompt can make a refusal disappear while removing the one instruction that mentions safety does not, because instructions are not independent constraints. Delta debugging guarantees a minimal failing set, not a culprit, so the search continues into pairs among the survivors — and the technique differs from context ablation in what it holds constant, what it confounds, and what a positive result licenses you to change.

   Read chapter
7. 7

   Promoting a trace into a change

   Three prompt changes, each verified by re-running the trace that prompted it, cost four points of aggregate quality at the next release: every new instruction is global state that interacts with everything else, while nothing checked the cases nobody complained about. The gate that prevents it turns on a class rate — construct fifteen siblings of the failing input and measure how many fail — which is also what separates a tail sample from a trace that represents an entire input class.

   Read chapter