RAG anti-patterns

Read these first

• RAG-specific metrics
• Hybrid search

What you will know

• Build a small RAG fixture that demonstrates over-chunking, stuffing k=20, missing reranking, embedding everything, and ignoring lexical search

  Use a fixed query set and inspect which documents enter the context, since plausible answers can conceal wasted tokens and missed exact matches.

• Measure retrieval and answer quality before and after removing at least two of the listed anti-patterns

  Report metrics such as recall@k, precision of the context, groundedness, latency, and token cost rather than relying on anecdotal improvements.

• Audit an existing RAG system against them

  Trace the path from query rewriting through retrieval, ranking, context assembly, and generation so hidden defaults are included in the audit.

• Find one in your own build

  Capture a failing query, the relevant logs or traces, and the smallest change that confirms the suspected cause.

• Use the list as a review checklist for someone else's design

  Require evidence for chunk size, candidate count, reranking, embedding scope, lexical fallback, and evaluation coverage before approving the design.

Chapters

1. 1

   A fixture that reproduces the defect beats an anecdote about it

   Start by building the thing that fails: a small corpus, a fixed query set, and switches that let you turn on 128-token chunks, k=20 without reranking, a dense-only index, and an ingest that swallows everything. The habit that matters is inspecting which documents actually entered the context, because a plausible answer hides both the wasted tokens and the exact match that never arrived.

   Read chapter
2. 2

   Groundedness stays high when nineteen of twenty chunks do not belong

   A dashboard reads recall@20 of 0.94 and groundedness of 0.91, and users still report that answers address the wrong thing. Both are satisfied by a context that is 5% signal: recall asks only whether the right chunk is somewhere in the twenty, and groundedness asks only whether the answer is faithful to what was retrieved. Context precision is the number nobody plotted, and cutting k to 5 without a reranker trades one defect for another. Naming the fix includes pricing it: a reranker is another hop, cross-encoder latency that grows with the k you feed it, and one more service to keep running.

   Read chapter
3. 3

   A 128-token chunk cuts a procedure into steps that arrive alone

   Smaller chunks were supposed to raise precision. Instead 'how do I rotate the signing key?' returns the one step whose wording matches the question, while the preconditions in the surrounding steps score lower and never arrive. You will trace that mechanism, then design the smallest experiment that confirms it — retrieval-side, before any re-embedding — and name the confounds that make a full re-chunk uninterpretable. An inconclusive result is itself a finding to read: with complaints concentrated on a handful of procedural queries, say whether the set was ever large enough to show the effect, separate 'not the cause' from 'not enough evidence', and name the hypothesis you test next.

   Read chapter
4. 4

   A reranker cannot fix an index that holds last year's policy

   Answers keep citing a policy replaced last year, and someone proposes a reranker. It will not help: a superseded document is genuinely relevant, and relevance is all a reranker orders by. The fix lives further upstream in ingest scope and metadata — excluding old versions, or attaching status and effective dates you can filter on — which is why 'embed everything' is a decision, not a default.

   Read chapter
5. 5

   Dense retrieval has nothing to match an error code against

   A runbook containing E_2231_TLSHANDSHAKE is indexed, the user searches for exactly that string, and a dense-only system returns nothing useful. Identifiers, error codes and proper names carry no semantic neighbourhood to embed, so the cheapest effective fix is a lexical retriever fused with the dense one. This chapter says when that fusion is worth its operational cost and when a metadata filter would do.

   Read chapter
6. 6

   Every item on this list is the right design for some corpus

   The failures so far make these look like prohibitions, and they are not. 128-token chunks are correct when the units are short and self-contained — FAQ entries, log lines, product attributes. k=20 with no reranker is correct for a recall-dominated task with a cheap long-context reader and no precision-sensitive generation. Dense-only is correct when the query distribution carries no identifiers, error codes or proper names. Ingesting everything is correct on a small, curated, single-version corpus. What makes each one a defect is not the setting but a team asserting it without ever measuring whether their corpus meets the condition.

   Read chapter
7. 7

   Removing two anti-patterns at once needs the pair of numbers

   Adding a reranker and cutting k from 20 to 5 are two changes, and one number cannot judge them. Report recall@k and context precision on a fixed query set before and after, with latency and tokens per query beside them, because a precision gain that quietly loses recall reads as success on half the evidence. Attribution is what before-and-after buys you.

   Read chapter
8. 8

   Hidden defaults live in the trace, not in the design document

   Ask for a full trace of one real production query — rewriting, retrieval, ranking, context assembly, generation — and you will find what the design document never mentions: the framework's default k, a similarity threshold silently discarding candidates, a chunk size inherited from a tutorial. The chapter turns that trace into an audit, then narrows a single failing query in your own system to the smallest change that confirms the cause.

   Read chapter
9. 9

   Wrong and unjustified are different review verdicts

   Blocking every item on the list is how a review becomes theatre: several of these designs are defensible on the right corpus, and what makes them defects is that a document asserts them without measurement. You will separate wrong from unjustified, state the evidence that clears each blocker, decide what to require when the team ships in three weeks, and explain why a gate everyone passes leaves complaints unchanged.

   Read chapter