Context engineering — eviction and compaction

Read these first

• Context engineering — selection

What you will know

• Compact a long conversation without losing what the next turn needs

  Preserve actionable commitments, unresolved questions, constraints, and references; summaries can otherwise introduce omissions or false continuity.

• Choose between a rolling summary and dropping turns, and justify it

  Dropping preserves exact surviving evidence but loses continuity, while summaries preserve continuity at the cost of drift and compression errors.

• Show what a compaction lost by testing before and after

  Compare task success, factual consistency, retained constraints, and prompt size on the same conversations rather than treating fewer tokens as success.

• Decide what must never be compacted away

  System and developer instructions, active tool state, safety constraints, user commitments, and unresolved decisions often need verbatim retention.

• Trigger compaction on a budget rather than a turn count

  Reserve room for the expected completion and account for tokenizer-specific limits, since a single large tool result can exhaust the context window.

Chapters

1. 1

   Dropping turns and rolling summaries

   The two standard ways of getting a long transcript back under budget preserve opposite things. Dropping the oldest turns keeps every surviving turn byte-exact and loses the rest loudly; a rolling summary keeps continuity across the whole span and loses exactness quietly. This chapter measures both against a 41-turn refund negotiation whose value is one agreed figure and two granted exceptions, derives the hybrid that pins those items verbatim while summarising the discussion around them, and gives the workload where the right answer is to run neither.

   Read chapter
2. 2

   Pinning constraints and commitments

   A transcript holds two kinds of thing: records of what happened, which may be paraphrased, and standing conditions that bind every turn after them, which may not. Instructions, safety constraints, active tool state, user commitments and unresolved decisions belong in a pinned ledger carried verbatim, and the classification is cheap and reliable at the moment each turn arrives and expensive and unreliable at the compaction threshold. This chapter draws the line, measures what a better summariser prompt buys instead, and covers retirement rules so the ledger does not become the budget problem.

   Read chapter
3. 3

   The agent state document

   A compaction output is not a document for a person catching up on a session; it is the agent state the next turn will reason from, and the two have different contents even when they describe the same three hours. A fluent narrative records that searches happened and not what each returned or what remains open, which is why an agent re-runs work after every compaction. This chapter gives the four sections a state document carries, the rule that payloads leave and keys stay, and the cases where a narrative is the better artefact after all.

   Read chapter
4. 4

   Hierarchical compaction

   Feeding the previous summary plus the new turns to the summariser is the cheapest design and the one that turns a question into an approval over six passes, because each pass is faithful to its input and no pass can see the original. Keep every turn in a durable log, summarise fixed spans exactly once from that log, and rebuild larger spans from the originals rather than from the summaries. This chapter prices the difference, places the block boundaries, and covers the retention rule that rules the design out for some products.

   Read chapter
5. 5

   The compaction trigger

   A turn counter encodes an assumption nobody wrote down: that the token cost of a turn is bounded and roughly constant. One tool that can return 45,000 tokens in a single result invalidates it, and compacting more often does not help because the oversized result is already in the failing request. This chapter replaces the counter with a measured budget that reserves the completion and a plausible next tool result, covers the three ways of measuring what is used, and puts a clamp at the tool boundary where the tool's own semantics can do better than truncation.

   Read chapter
6. 6

   Evaluating a compaction change

   Prompt size down 55%, task success from 0.83 to 0.84 over 200 replayed conversations, and much better-written summaries: none of that is evidence a compaction change is safe. Dropping one safety constraint or inverting one user commitment lands in a handful of conversations and cannot shift a mean. This chapter builds the per-item test — a fixture naming what must survive, a substring check for pinned text and a three-way judge verdict for the rest — with token savings reported alongside rather than as the result.

   Read chapter
7. 7

   Compacting a ninety-minute session

   A coding agent sits 8,152 tokens over budget holding repo operating constraints, a prohibition stated at turn 3, a five-step plan with two steps done, a test run from two turns ago and sixty turns of reads and edits. This chapter works the real decision item by item — what is summarised, what is pinned verbatim, what is evicted to a reference, and what trace the compaction leaves of what it removed — then changes the design so the call stops arriving 8,000 over, and gives the order in which to adopt any of it.

   Read chapter