Anatomy of a prompt

Read these first

• System prompts

What you will know

• Write a prompt with a clear task, constraints, and output format

  A reviewer or parser should be able to determine what success means and reject outputs that violate the required shape.

• Identify which of those four parts is missing from a prompt that underperforms

  Separate an absent instruction from conflicting requirements, irrelevant context, or model variance before changing the prompt.

• Rewrite a vague prompt into a specific one and measure the difference

  Use a fixed evaluation set and a stated quality metric, because a single appealing response cannot establish an improvement.

• Trace a bad answer to a role, task, context, constraint, format, or example change using a fixed evaluation set

  Small changes can interact, and an example may override a written rule or anchor the model to an unintended pattern.

• Cut a prompt in half without losing quality

  Track tokens, latency, and evaluation scores while removing redundancy; shorter context can help, but deleting a crucial example or constraint can cause regressions.

Chapters

1. 1

   The six parts of a prompt

   Whatever a prompt looks like on the surface, it answers six questions on the model's behalf: who is answering, what they are doing, what they know, what they may not do, what the reply must look like, and what a correct one looks like in practice. Reading a prompt as six slots turns an argument about prose into an argument about which slot is absent — and naming an output format is what creates the category of an invalid reply, which is what lets a parse failure be counted instead of silently producing nothing.

   Read chapter
2. 2

   Diagnosing a missing part

   Replies that vary in shape from call to call, open with 'Certainly!', or return a category outside your list are not general quality problems, and each points at one absent slot. Shape drift means no output format was stated; out-of-vocabulary values mean no constraint; a wandering answer often means context nobody needed. Sorting thirty failures into buckets takes twenty minutes and replaces a mood with a count — which is what tells you that two sentences address 97 of 116 failures and a week of new examples addresses them by suggestion.

   Read chapter
3. 3

   Examples that contradict constraints

   Adding three well-written few-shot examples to a prompt dropped compliance with an eight-month-old constraint from 96% to 61%, with the rule still there and unedited. Examples are read as the operative specification of a correct output, so a demonstration that breaks a stated rule wins over the sentence stating it, and a nine-line unit test catches the contradiction at commit time. The same property has a second edge: an example only ever evidences the values it happens to contain.

   Read chapter
4. 4

   Conflicting requirements in a prompt

   'Answer in at most three sentences' and 'cite every claim inline' are individually reasonable and jointly unsatisfiable past two claims, so answers either run long or drop citations, roughly half and half, and strengthening either rule makes the other worse. Three signatures separate a conflict from an absence — the seesaw, bimodal failures, and a failure rate that tracks a property of the input — and the three repairs are declaring a winner, moving one requirement outside the other's budget, and scaling the budget.

   Read chapter
5. 5

   Measuring a prompt change

   A claim that prompt B beats prompt A needs four things: cases fixed and held out, decoding settings pinned to an exact model version, a metric stated and applied identically to both, and a gap larger than the noise. Three runs of an unchanged prompt on 120 cases came back 71.7, 72.5 and 70.8, so anything under 1.7 points is nothing. When a four-change commit dropped twelve points, re-applying the changes one at a time found nine of them in a single swapped example.

   Read chapter
6. 6

   Shortening a prompt

   Instructions covering rare conditions look identical to instructions covering nothing, which is why cutting a 1,400-token prompt to 640 dropped an eval from 84 to 76: a 25-token clause was the only place one behaviour was ever specified. What goes first, what you refuse to delete without a slice score, why leave-one-out ablation is blind to a rule stated twice, and why prompt caching usually beats cutting on cost — plus how an aggregate moving 0.4 points hides a slice falling off a cliff.

   Read chapter
7. 7

   Versioning a prompt

   A prompt edited in place has no sequence of versions to bisect and no scored artefact to roll back to, so a quarter of gradual degradation cannot be attributed or undone. The destination is prompts in the repo, but review alone buys the appearance of control — nobody can tell by reading a diff that a swapped example cost twelve points. The gate that actually works is a 200-case eval posting a per-slice table on every pull request, plus an expiring override so an emergency fix still ships in 30 seconds.

   Read chapter