Long-horizon code tasks

Read these first

• Evaluating coding agents
• Planning

What you will know

• Decompose a large code task into verifiable steps

  Each step should produce an observable artifact or test result, with dependencies and rollback boundaries made explicit.

• Run an agent on a multi-step change with a persisted task ledger and resume it after interruption

  The workflow must recover from context loss without repeating completed edits or silently skipping unfinished work.

• Instrument a long-horizon coding run for progress, token usage, tool failures, and validation outcomes

  These signals expose whether extra iterations are buying correctness or merely increasing spend and latency.

• Reproduce a thrashing run and patch its stopping or recovery policy

  Repeated equivalent tool calls, unchanged diffs, and failing tests should trigger a bounded recovery path rather than more of the same.

• Write and test a handoff policy that routes blocked work to a human with sufficient context

  A useful handoff includes the current state, attempted evidence, unresolved risks, and the smallest decision the human must make.

Chapters

1. 1

   A step nothing can verify is a failure nobody can attribute

   A forty-call-site migration is planned in six phases, and phase three — update the shared wrapper and adjust callers — cannot compile or pass tests until phase five lands. When the run fails after phase four, nothing can say which phase broke it, and the rollback boundary is the entire migration. Build steps that each end in an observable artifact and a named check, while staying honest about when three coarse steps beat twelve fine ones. A step with no check that separates success from failure has two honest fates: fold it into the neighbour whose check does cover it and accept the wider rollback boundary, or manufacture a check — a characterisation test pinned to current behaviour, a temporary assertion removed when the plan completes, a snapshot of the artifact it is supposed to produce.

   Read chapter
2. 2

   Progress that lives in the conversation dies with the conversation

   A multi-step refactor killed at 70% by a wall-clock limit resumes by re-applying edits that had already landed and skipping one that had not, because its only record of progress was context it no longer has. The ledger that fixes this holds more than a step list and a done flag: the paths each step touched, the check that certifies it, its dependencies, and its rollback boundary, written somewhere that outlives the run.

   Read chapter
3. 3

   A ledger records intent; only the tree records what happened

   A run killed between applying an edit and updating its flag is misrecorded in whichever direction the write ordering happens to produce, so a status field read on resume is a claim rather than a fact. Reconstruct status by re-running each step's certifying check against the working tree instead. You will also see what a step interrupted halfway through its edits leaves behind, and precisely what the resumed run should observe when it looks.

   Read chapter
4. 4

   The ledger is working memory, not only crash recovery

   A run that spends 380k tokens over 22 iterations has lost its early context with nobody killing it, so the reconstruction that serves a resume has to happen at the start of every step, not only after a failure. Read back the remaining plan, the ledger entries with their certifying checks, the current state of the tree and the failure record from the last attempt — and deliberately leave out the transcript those entries were distilled from, or the window refills with the very material the ledger exists to replace. A ledger consulted only on resume leaves the model improvising from whatever survived compaction.

   Read chapter
5. 5

   A diff that keeps changing is activity, not progress

   A run has consumed 380k tokens across 22 iterations and is still going; the total is not the number that decides anything. Instrument the trajectory instead — validation outcomes per iteration, whether failures are decreasing, recurring or newly appearing, how much the diff actually moved, tool failure rates and spend per attempt. Read together, these separate iterations buying correctness from iterations buying a larger diff for somebody to unpick.

   Read chapter
6. 6

   Detect the unmoving failure set, recover once, then stop

   The clearest thrash signature is the same tool call producing the same failure repeatedly; the costlier one is an identical failing check set across four attempts whose diffs keep changing substantially. Both say the loop lacks something more sampling will not supply, so raising the attempt limit buys spend and confusion. Design the detector, then the bounded recovery it triggers — re-read the tree, re-plan a narrower step, one attempt under the new framing — and name the legitimate run it would wrongly halt.

   Read chapter
7. 7

   A handoff is a decision request, not a transcript

   When the stopping policy fires at 3am, the engineer who opens it should find the smallest decision they actually have to make stated first, then the current state of the tree, what was attempted with the evidence each attempt produced, and the risks left unresolved. Say explicitly what the run did with the work — reverted to the last certified step, or preserved on a branch and named — because the state the human inherits is the largest cost in the handoff. The chapter closes on the harder question: whether longer autonomous horizons are worth chasing when the return on horizon is capped by the strength of the per-step checks, and whether shorter supervised cycles, better verification tooling or several short runs in parallel buy more than one long one.

   Read chapter