Prompt decomposition

Read these first

• Anatomy of a prompt

What you will know

• Split a failing mega-prompt into two prompts that each work

  Independent prompts can still fail at the handoff when schemas, context selection, or instructions are underspecified.

• Measure the added latency, token use, and cost

  A second call can make a cheap request materially slower and more expensive, especially when it repeats large context.

• Evaluate each step independently

  Per-step pass rates and error labels reveal whether a later stage amplified an upstream defect.

• Decide when the extra call is not worth it

  For a simple task, the added round trip may deliver no quality gain and becomes pure operational overhead.

• Trace a prompt that is doing three jobs badly

  Unrelated outputs often compete for context and instructions, making both quality regressions and ownership harder to diagnose.

Chapters

1. 1

   Score each job separately before you split the prompt doing three

   One prompt condenses each support ticket, assigns a priority from four levels and drafts a reply, and the priorities look erratic. Splitting all three now makes two explanations permanently indistinguishable: the jobs competing for attention and context, or priority rules that were never written down properly. This chapter covers scoring each output against its own criteria first, which is what tells you whether decomposition is the right instrument.

   Read chapter
2. 2

   Two calls sharing one document double the input and add the latency

   Both halves of a split need the same 4,000-token contract in context and each returns about 150 tokens, so input tokens roughly double and the calls run strictly sequentially rather than overlapping. You will learn to price a decomposition before proposing it: added input spend at real volume, a p95 moving from 1.4s to something near 2.5s, and which parts of the bill do not change at all.

   Read chapter
3. 3

   A second call that learns nothing the first already knew

   Turning a one-line request into a catalogue search query already fits a 300ms budget at 240ms. Splitting it into 'detect intent' then 'write query given the intent' turns one round trip into two for a step that receives no information the first call did not already have. Covers recognising that shape and refusing the split without becoming hostile to decomposition in general.

   Read chapter
4. 4

   A stage that retrieval or a rule can do needs no model call

   Accepting a split and refusing it are not the only two options. When the proposal is 'retrieve the relevant sections, then answer', the selection step is usually better served by retrieval, deterministic chunking or a business rule than by a second model call, and sometimes the whole benefit comes from restructuring the single prompt so the relevant material is surfaced inside it. This chapter covers recognising the stages that do not need a model at all, what each substitute costs in engineering and in flexibility, and how having a third option changes the answer to 'is the extra call worth it'.

   Read chapter
5. 5

   Per-step pass rates plus a fault label for every failure

   A three-step pipeline at 82% with nobody able to name the step to work on is an instrumentation gap rather than a modelling problem. You will learn to record per-step pass rates plus a label on every failure saying whether that step failed on a good input of its own or faithfully passed along a bad one from upstream, which is the distinction that locates the losses.

   Read chapter
6. 6

   95% and 97% steps producing a 79% pipeline: the handoff

   Step one passes its own eval at 95% and step two at 97%, yet end to end you measure 79%, well below what independence predicts. The gap is a signal, not noise: step two was scored on hand-written gold inputs it never sees in production, so it confidently validates a plausible-looking wrong value. Covers evaluating a step on its predecessor's real output distribution.

   Read chapter
7. 7

   Step two cannot fetch what step one's condensation threw away

   When step one condenses a 6,000-word policy document and step two answers using only that condensation, accuracy can fall below the single mega-prompt it replaced. The mechanism is not a formatting failure at the boundary: step one dropped the detail step two needed, and step two has no route back to the source. Covers designing the handoff as a contract rather than discovering it as a bug — what the intermediate carries, whether it is a validated schema, whether step two also receives the source document, and what step two does when the intermediate arrives malformed.

   Read chapter
8. 8

   Eleven prompts, 68% accurate, and a twelfth after every incident

   A document pipeline of eleven chained prompts scores 68% end to end at 9s p95, and each incident review adds another step. This chapter argues both sides: what the design genuinely buys in attribution and ownership, why the losses compound and the latencies add, and how to propose a change to a team that treats small focused prompts as settled doctrine without substituting one untested design for another.

   Read chapter