The tail restatement

The tail restatement is a few dozen tokens appended after everything else in the prompt, naming rules that already exist in the policy at the front. It is not a second policy and it carries no new normative content: it is a pointer, positioned where the model reads most reliably, at a cost of tens of tokens per request against a prefix worth thousands.

The reason it exists is the trade the previous chapters have been pricing. A stable-first layout keeps the prefix and demotes every rule in the prompt; a documents-first layout promotes the rules and destroys the prefix. The restatement is the third option, and the case for it is measurable on a single sheet — compliance predicates, hit rate, and the input bill, all from one case list. The figures below come from an account agent running 12,000 tokens of immutable material at 120,000 requests a day, at $3 per million input tokens.

Fifty-eight tokens after the user's turn#

That agent's prompt is the policy, the escalation matrix, the tone guidance and the schemas for nine tools, three of which change customer state. Behind it sit the retrieved documents, capped at six chunks, and the user's turn. The layout holds the prefix and it costs compliance on three rules the business enforces — every factual claim carries a source link, no answer states a price, and no state-changing tool fires without an explicit confirmation from the user. On the gate from chapter 4 those three predicates score 92.0%, 99.4% and 99.2%, against floors of 98.5%, 99.5% and 99.9%. All three fail.

The repair is 58 tokens appended after the user's turn:

Before answering, apply these three rules from the policy above:
1. Every factual claim carries a source link from the documents provided.
2. Never state a price, a discount, or a contract value in your answer.
3. Never call close_account, issue_refund or change_plan without first
   asking the user to confirm in their own words.

Re-running the same 400 cases with that block in place moves the three predicates to 98.9%, 99.7% and 99.8%. Check each against its floor rather than reading the direction of travel: 98.9 clears 98.5, 99.7 clears 99.5, and 99.8 does not clear 99.9. Two pass and one fails, which is a result worth holding onto rather than rounding off — the third rule is the subject of the next chapter precisely because it did not clear.

The prefix is untouched, and that is a structural claim rather than a hopeful one. The 58 tokens are appended after the user's turn, which is behind the breakpoint, and prefix matching only cares about tokens in front of the breakpoint. Nothing in front of it moved, so the stored entry still matches, so the hit rate stays at 89% and the input bill stays where it was plus the cost of 58 uncached tokens.

Where the restatement goes#

The block belongs at the very end of the assembled prompt, after the user's message, because the position that scored 99.4% in chapter 2's placement probe was the last position and every token added after a rule pushes that rule back up the curve. In a Messages-shaped API, meaning one where a request is a system field followed by an ordered list of role-tagged messages, that means a second content block on the final user message rather than a second system block, since anything added to system lands ahead of the conversation.

The difference between the plausible positions is large enough to be worth measuring rather than reasoning about, and the same 400 cases give it directly:

The first row is worth more attention than the third. Putting the reminder immediately behind the policy takes citation compliance from 92.0% to 93.1%, which is 1.1 points for the same 58 tokens the third row spends to buy 6.9 — close enough to nothing that a team measuring only that arrangement would conclude the technique does not work. Same words, same length, same policy. The only thing that differs between the first row and the third is how many tokens sit behind the block, which is the clearest available demonstration that the words are not what is doing the work.

The last row prices how quickly that advantage decays. Appending 400 tokens of formatting guidance behind the reminder takes it from 98.9% back to 96.2%, giving up nearly three of the 6.9 points the reminder had bought — about a third of it — for 400 tokens. Put that beside the roughly 2,600 tokens of documents and user turn that already sit between the policy and the end of the prompt: 400 tokens is a small fraction of that distance and it costs a third of the gain, so the curve is steep near the end and shallow further back. That is why "last" is a position with a short half-life. The reminder has to be genuinely last, not nearly last, and a few hundred tokens is enough to stop being either.

{
  "model": "claude-sonnet-4-6",
  "system": [
    {
      "type": "text",
      "text": "You are the account agent for Northwind...\n<12,000 tokens of policy, escalation matrix, tone>",
      "cache_control": {"type": "ephemeral"}
    }
  ],
  "tools": ["...nine schemas, serialized ahead of the breakpoint..."],
  "messages": [
    {
      "role": "user",
      "content": [
        {"type": "text", "text": "<documents>\n...six retrieved chunks, ~2,400 tokens...\n</documents>"},
        {"type": "text", "text": "Can you close the Fenwick account and refund the last invoice?"},
        {"type": "text", "text": "Before answering, apply these three rules from the policy above:\n1. ...\n2. ...\n3. ..."}
      ]
    }
  ]
}

The breakpoint sits at the end of the system block, which is also the end of everything invariant, so the 12,000 tokens in front of it are the cacheable segment and the roughly 2,600 tokens behind it are charged fresh on every call. Serializing the tool schemas before the breakpoint rather than after is what keeps them inside the segment, and the reason is the one from chapter 1 applied to a marker instead of to a divergence: the cacheable region runs from position zero to the breakpoint, so membership is decided by position and by nothing else. A schema block is invariant, it never changes, and if it is serialized after the marked content it is outside the region and billed at full rate on every request anyway. "Stable" is not a property that earns a discount. "Stable and in front of the marker" is.

The tail has to be bounded for this structure to hold its shape. Chapter 4's citation regression was a function of distance — the rule at token 7,300 with six documents behind it — and a retrieval layer that occasionally returns fourteen chunks instead of six moves the restatement from 2,600 tokens after the policy to 9,000 tokens after it. The restatement is still last, so it still benefits from recency — the reliability advantage that material gets from sitting nearest the end of the prompt — but the policy it points back to is now much further away, and rules that depend on detail in that policy degrade even when the pointer does not. Capping the tail is the same discipline topic 4.13 applies to the budget as a whole, arriving here as a layout requirement: the tail's maximum size is part of the layout, and a retrieval change that raises it is a layout change.

One copy per request, not one per turn#

The account agent is conversational, and the obvious implementation appends the reminder to each user message as it arrives, which means turn eight carries eight copies scattered through the history at 58 tokens each. Chapter 2's turn-index measurement already explains why the compliance figure decays across a session. The duplicated reminder makes it worse rather than better, and the mechanism is the same one that sank the repeat-everything row: seven of the eight copies are now interior text, and the eighth — the only one in the position that was ever doing the work — is the same 58 tokens it always was, now arriving after seven earlier statements of the same rules that the model has already produced answers alongside. Scored on turn-eight cases, the accumulating version reached 95.1% citation compliance against 98.4% for the version that carries exactly one copy. Adding copies of a rule made the rule less well followed, which is the opposite of what anyone implementing it expects and follows directly from position being the thing that matters.

TAIL_MARKER = "<rules-reminder>"

def with_tail_restatement(messages, block):
    """Exactly one copy, on the newest user message, none anywhere in history."""
    cleaned = [strip_blocks(m, TAIL_MARKER) for m in messages]
    last = cleaned[-1]
    assert last["role"] == "user", "the restatement follows the user's turn"
    last["content"].append({"type": "text", "text": f"{TAIL_MARKER}{block}</rules-reminder>"})
    return cleaned

Stripping earlier copies has a cache consequence if you are also marking a breakpoint at the end of the conversation history to reuse the turns already exchanged, which is the standard way to keep a long session cheap. Rewriting turn three to remove a reminder invalidates every entry from turn three onward, so the reminder has to be attached after the history is serialized and removed before the next turn is appended to it, keeping the mutable element strictly behind the last breakpoint. That is the same discipline as the prefix, applied to a second boundary further down the prompt.

The tool loop needs the same treatment for a less obvious reason, and the sequence is worth walking because it happens after the code that built the prompt has finished running. The request is assembled with the reminder last. The model calls a tool. The result is appended to the conversation, so the prompt is sent again with the tool_result after the reminder. The model calls another tool, and another result goes on the end. By the time an answer is written, the reminder is two or three tool exchanges from the last position, and nothing in the assembly code did anything wrong — the prompt simply grew after assembly finished. Re-attaching the block after the newest tool result on every iteration of the loop took citation compliance on answers that followed three tool calls from 96.0% to 98.7%. The rule that generalises is that the reminder is positioned relative to the end of the prompt as it exists at generation time, not relative to the user's turn as it existed when the request was assembled.

What fifty-eight tokens cost, and what they bought#

Four structures, one case list, one model version, scored on the same afternoon:

The second row is the version teams reach for first and it is the worst option on the sheet, which is the result worth understanding rather than the one worth memorising. The reasoning that produces it is sound as far as it goes: the restatement works because the rules are at the end, more of the policy at the end should therefore be better, and repeating all of it puts every rule at the end. What that misses is that "the end" is not a region a block can be moved into. It is a single position, and it is occupied by exactly one thing. Append 12,000 tokens and the last of them is at the end while the other eleven thousand-odd are in front of it, so the three rules that matter are once again buried in the middle of a long block — this time the middle of the second copy. The predicates confirm it: 97.4%, 99.6% and 99.5%, every one of them below the 58-token version. Restating everything restates nothing.

It is also the most expensive row apart from giving up the prefix. Against the first row it adds $4,062 a day, and the reason that figure is so much larger than the restatement's is the token column beside it rather than anything subtle: 12,000 extra tokens per request against 58 is more than two hundred times as many, charged the same way, on the same traffic.

The third row is the cheap one, and its cost is worth deriving because the number looks too small to be right. Each request carries 58 extra tokens. There are 120,000 requests a day. That is a little under seven million extra tokens a day, and seven million tokens at $3 per million is about $21 — which is why the third row's daily spend is the first row's $1,979 plus twenty-odd dollars. What those $21 buy is 6.9 points of citation compliance, and applying that percentage-point gain to 120,000 requests gives about 8,300 answers a day that now carry their sources rather than not.

The comparison that matters is against the fourth row. Moving everything after the documents scores 99.2% against the restatement's 98.9%, so it buys 0.3 points more, and it costs $4,421 a day against $2,000, so it costs $2,421 a day more. Both of those subtractions are on the same sheet and neither of them was available while the argument was between two layouts. Once the restatement is on the table, the original dilemma is no longer between an 89% hit rate and reliable citations.

Imperatives travel, procedures do not#

The restatement does not carry procedure. "Ask before destructive actions" fits in a line; the escalation matrix that decides which of four teams a refund above $5,000 routes to does not, and shortening it into the tail produces a rule the model can follow incorrectly with confidence. Rules that compress to an imperative belong in the tail; rules that are lookups belong in the prefix, and if the model is failing at them the fix is elsewhere.

It does not raise a rule's ceiling either. The tail restatement got citations to 98.9%, which is the same neighbourhood as the policy-last layout's 99.2% and no better, because both are measuring what the model does with the rule when it is in a good position. If a rule is at 91% because it is ambiguous, or because it conflicts with a demonstration elsewhere in the prompt, moving it does not resolve the ambiguity, and a placement fix applied to an authoring problem produces a small improvement that hides a large one.

And it is not a control. A rule that must hold on every qualifying response — a regulatory disclaimer, a hard block on a tool call — needs enforcement outside the model, in a validator that rejects or repairs the response, or in a permission check in the tool handler. The tail restatement moves a rate; it does not create a guarantee, and the number of nines you need decides which of those you are shopping for.

Within those limits it is the cheapest lever in this topic by a wide margin. A few dozen tokens at the end of the prompt, generated from the policy rather than typed beside it, attached to whatever is genuinely last at generation time and held to three or four rules, recovers most of a stable-first layout's compliance cost while the prefix carries on being read at a tenth of the base rate. Both of the expensive answers — repeat everything, or give up the prefix — score worse than that on this surface.

Two of the three predicates cleared their floors. The confirmation rule at 99.8% did not clear 99.9%, and it is the one attached to a tool that closes customer accounts. That leaves a decision the worked examples in this topic have so far avoided.