A failed step invalidates the steps that assumed it

An agent run built out of the structured steps of the previous chapter can fail in the middle. Step three calls a tool, the tool refuses, the executor has a decision to make, and the decision it makes by default in most implementations is to record the failure and carry on to step four. This chapter is about why carrying on is unsound in general rather than unlucky in this case, and about what a recovery path has to be given before it can do anything useful.

The setting is a churn close-out queue, planned as a batch. At 23:50 each night a job takes the accounts that finished their notice period and asks the planner for a six-step plan per account; the executor works through the queue starting at 07:30 the next morning. That gap between planning and execution matters later and it is worth noticing now: for account acct_3319f the plan was written at 23:50 on 11 March, the data-retention job ran at 02:14 and archived the customer record because the account had been inactive for more than 400 days, and execution began at 07:31:08. Subtract the two clock times and the plan was seven hours and forty-one minutes old when the first tool call went out.

07:31:08  step 1  billing.list_invoices(account_id=acct_3319f)
          -> 1 draft invoice in_88a2, €612.40; billing_contact "ar@northwind-old.example"
          POST result_count>=0 ok  (bind: invoice_id, billing_contact, sub_id)
07:31:09  step 2  billing.void_invoice(invoice_id=in_88a2)
          -> {"status": "void"}                                    POST ok
07:31:09  step 3  customer.update(account_id=acct_3319f,
                                  status="closed", billing_contact="ar@northwind.example")
          -> 409 {"code": "record_archived", "archived_at": "2026-03-12T02:14:07Z"}
07:31:09  step 3  retry 1/3 after 200ms  -> 409 record_archived
07:31:10  step 3  retry 2/3 after 400ms  -> 409 record_archived
07:31:10  step 3  retry 3/3 after 800ms  -> 409 record_archived
07:31:10  step 3  retries exhausted; marking step failed, continuing
07:31:11  step 4  billing.cancel_subscription(sub_id=sub_71c2, at="period_end")
          -> {"status": "pending_cancellation"}                    POST ok
07:31:12  step 5  billing.issue_credit(account_id=acct_3319f, amount_eur=612.40)
          -> {"credit_id": "cr_4a90", "applied_to": "suspense"}    POST ok
07:31:12  step 6  email.send(to="${billing_contact}", template="account_closed")
          -> {"message_id": "m_5518"}                              POST ok
07:31:12  run acct_3319f: SUCCESS (6 steps, 1 failed, 5 succeeded)

Three retries cost 1.4 seconds of backoff — 200 milliseconds, then 400, then 800, doubling each time — and produced the same 409 each time, which is the part of this trace that draws attention in review. The expensive part is the four seconds after it. Steps four, five and six executed against a world in which the customer record was never closed and never re-pointed at the current accounts-receivable address, and all three of them succeeded, so the run reported success. A €612.40 credit sits in a suspense balance nobody reconciles. A subscription is pending cancellation on an account whose closure was never recorded, so the weekly churn report will not include it. And the closure email went to ar@northwind-old.example, which is the address step three existed to replace.

Why all three of them succeeded#

The steps that follow a failed step fall into two groups, and which group a step lands in is decided by how the planner happened to write its arguments rather than by anything about the work.

Every one of these depends on step three. What distinguishes them from a step that would have failed is not the strength of the dependency but its kind. A plan's ${...} references are value-flow edges: they say that this argument is that earlier result. The binder can see those, and a step whose argument references a step that produced nothing fails loudly, because there is nothing to substitute. Every other kind of dependency — this step should only run against a closed account, this credit is meant to reach the new contact — is a statement about the state of the world, and the plan does not contain it in any form a machine can read.

Step six is one substitution away from illustrating exactly that. Had the planner bound billing_contact from step three's response rather than from step one's, the run would have stopped with a bind error and somebody would have looked at it that morning. The address in question is the same address either way; only the JSON path differs. The difference between a loud failure and four seconds of silent damage was a JSON path, and no reviewer reading the plan would have flagged the choice, because both paths yield a valid email address on a successful run.

It is worth being concrete about what the assumption was, because "step four assumed step three" is the kind of phrase that sounds like an explanation and is not one. The close-out procedure this plan encodes exists so that the finance system and the CRM — the customer-relationship management system, which is where the account record and the billing contact live — agree: an account marked closed has no active subscription and no unapplied balance. Step three is the only step that touches the CRM side of that invariant, and steps four and five are the finance side.

So running the finance side without the CRM side does not produce a partially closed account. It produces two systems that disagree, which is the specific state the procedure was written to prevent, and it is worse than having done nothing at all — doing nothing leaves an account that is open in both systems, which is consistent and which the next night's batch would have picked up. That is why the run being five-of-six successful is not 83% of a good outcome. The steps are not independent contributions to a score; they are moves toward a single invariant, and the invariant either holds or does not.

That is also the reason "continue past a failed step" cannot be made safe by being careful about data dependencies. Suppose the executor were given a dependency graph and told to skip any step transitively downstream of a failure. It would skip nothing here, because the graph contains only value edges and none of steps four to six has one. The assumptions a plan carries are mostly not values, no static analysis of the plan will surface them, and the planner was never asked to write them down. The only sound reading of a failed step is that every step written after it was written under an assumption that no longer holds, so the remainder of the plan is void until something re-examines it.

The test that catches this#

The defect is stated precisely enough to test: after a step's failure is final, no step later in the plan is dispatched, and the run does not terminate in a success state.

def test_failed_step_voids_the_remainder():
    tools = FakeTools()
    tools.customer.update.raises(ToolError("record_archived", status=409, retryable=False))

    run = execute(plan_for("acct_3319f"), RunState(), actor=BATCH_ACTOR)

    assert run.status == "replanning"          # not "success", and not "partial"
    assert tools.calls("billing.cancel_subscription") == 0
    assert tools.calls("billing.issue_credit") == 0
    assert tools.calls("email.send") == 0
    assert run.failure == {"step": 3, "code": "record_archived", "retryable": False}
    assert run.completed_steps == [1, 2]

The three zero-call assertions are the substance, and the reason they have to be written as call counts rather than as a status check is worth being explicit about. A version of this test that only checks run.status != "success" passes against an executor that continues through steps four to six and then reports partial failure at the end. That executor causes this incident exactly as the original did, with a more honest label attached to the result. The behaviour under test is not what the run says at the end; it is what the run refrains from doing in the middle, and only a count of calls that did not happen expresses it.

"Final" is doing work in that sentence and deserves a definition, because the executor has to decide when to stop attempting before it can decide what to do about having stopped. A failure is final when the retry policy has no attempts left for it, or when the error is classified as one that repetition cannot change. Both conditions terminate the same way, in a failure record that the run state stores and the remainder of the plan is voided against. Keeping the two decisions separate — how many times to attempt, and what a final failure does to the plan — is what stops a retry-policy change from silently altering recovery behaviour.

The fix that gets proposed first is a retry policy that distinguishes transient errors from permanent ones, so that a 409 with record_archived is not retried at all while a 503 from the billing gateway is. That is worth doing and it is worth being exact about what it buys. Apply it to this run and the three retries disappear, so the run reaches its decision point 1.4 seconds sooner. Steps four, five and six then execute, the €612.40 still lands in suspense, the churn report still misses the account, and the run still reports success. Retry policy governs how many times you attempt an operation. Nothing in it governs what the rest of the plan does once you stop attempting, and the two are separate decisions precisely so that improving one cannot be mistaken for addressing the other.

What the re-planner has to be handed#

Stopping is only half of it, because the account still has to be closed. The response to a final failure is a second planner call, and the quality of what comes back is decided almost entirely by what that call is given.

Test each of the two inputs by imagining it absent, which is the quickest way to see what it is for.

Omit the completed work. The re-planner receives the goal and the failure, reasons about what closing out an account requires, and emits a plan that begins by voiding the open invoice — because from where it is sitting, that is step one of the procedure. Invoice in_88a2 is already void. billing.void_invoice on an already-void invoice returns a 400, which the executor reads as a new failure, so the run enters a second recovery for a problem the first recovery created. Completed work also carries the bindings, which is what lets the new plan reuse sub_71c2 and €612.40 instead of re-reading them from systems that may since have changed.

Omit the remaining work. The re-planner receives the goal, the completed steps and a failure record saying customer.update returned 409. It produces a plan to fix that, which unarchives the record and updates it, and stops — because the failure it was handed is the whole of the problem it was shown. The subscription is never cancelled. The run's obligation is the goal, and the remaining steps are the previous planner's best decomposition of what is left of it, worth passing as a proposal the new call may keep, amend or discard.

{
  "goal": "close out acct_3319f: void open invoices, close the customer record, cancel at period end, credit the unused balance, notify the billing contact",
  "completed": [
    {"n": 1, "tool": "billing.list_invoices", "effect": "read",
     "binds": {"invoice_id": "in_88a2", "sub_id": "sub_71c2",
               "billing_contact": "ar@northwind-old.example", "amount_eur": 612.40}},
    {"n": 2, "tool": "billing.void_invoice", "effect": "write",
     "args": {"invoice_id": "in_88a2"}, "result": {"status": "void"}}
  ],
  "failure": {"n": 3, "tool": "customer.update", "code": "record_archived",
              "http": 409, "retryable": false,
              "detail": {"archived_at": "2026-03-12T02:14:07Z"}},
  "remaining": [
    {"n": 3, "tool": "customer.update", "args": {"account_id": "acct_3319f",
      "status": "closed", "billing_contact": "ar@northwind.example"}},
    {"n": 4, "tool": "billing.cancel_subscription", "args": {"sub_id": "${sub_id}"}},
    {"n": 5, "tool": "billing.issue_credit", "args": {"account_id": "acct_3319f",
      "amount_eur": "${amount_eur}"}},
    {"n": 6, "tool": "email.send", "args": {"to": "ar@northwind.example"}}
  ],
  "budget": {"replans_used": 0, "replans_max": 2}
}

What comes back is a plan that treats the completed work as done and the obstacle as a thing to route around.

{"steps": [
  {"n": 1, "tool": "customer.restore", "args": {"account_id": "acct_3319f"},
   "post": {"probe": "field_equals", "path": "$.archived", "value": false},
   "on_unmet": "halt"},
  {"n": 2, "tool": "customer.update",
   "args": {"account_id": "acct_3319f", "status": "closed",
            "billing_contact": "ar@northwind.example"},
   "post": {"probe": "field_equals", "path": "$.status", "value": "closed"},
   "on_unmet": "replan"},
  {"n": 3, "tool": "billing.cancel_subscription", "args": {"sub_id": "sub_71c2"},
   "post": {"probe": "field_equals", "path": "$.status", "value": "pending_cancellation"},
   "on_unmet": "replan"},
  {"n": 4, "tool": "billing.issue_credit",
   "args": {"account_id": "acct_3319f", "amount_eur": 612.40},
   "post": {"probe": "field_not_equals", "path": "$.applied_to", "value": "suspense"},
   "on_unmet": "halt"},
  {"n": 5, "tool": "email.send",
   "args": {"to": "ar@northwind.example", "template": "account_closed"},
   "post": {"probe": "field_present", "path": "$.message_id"}, "on_unmet": "retry"}
]}

Three details in it are worth reading rather than skimming. No void step appears, because the completed record said it was done. The credit step now carries a postcondition asserting the credit did not land in suspense, which is the check that would have caught the March run at the moment it went wrong — the original plan's credit step had a postcondition that the call succeeded, and the call did succeed; what it failed to assert is where the money went. And customer.restore fails to halt rather than to replan, since a plan that cannot restore the record has no route to the goal, and asking the planner again for a route that does not exist produces invention rather than recovery.

Recovery also needs a ceiling. replans_max: 2 exists because a planner handed the same obstacle three times will produce three plans that fail identically, at roughly 3,400 input tokens each; the budget converts a loop into an escalation.

Notice what re-planning did not do. Nothing un-voided invoice in_88a2, and nothing was going to, because the void is a real state change in a real billing system and the recovery path has no authority to reverse it. That gives the mechanism a precise licence and a precise residual. What it licenses is a route from the world as the completed steps left it to the goal — forward-only recovery, which is a weaker guarantee than a transaction and is the guarantee available when the steps are calls to systems you do not control. What it leaves out is any claim that the intermediate states were safe: the plan may pass through a world in which an invoice is void and an account is open, and if that combination is itself harmful, no amount of re-planning helps. It also means the completed record has to be accurate rather than optimistic, since the new plan is built on the assumption that everything listed there actually happened. A step whose outcome is genuinely unknown breaks that assumption, which is a case the fourth chapter has to deal with directly.

Where preconditions fit, and what they leave alone#

A postcondition asks whether a step reached the state it aimed for. Its mirror is a precondition, asking before the call whether the assumptions the step was written under still hold. Topic 10.5 covered how to build these so they read the world rather than the plan, and the reason to raise them again here is that they look like a complete answer to this chapter's incident and are not one.

They do fix something real. A precondition on step three — the customer record exists and is not archived — turns a 409 during a write into a check that fails before any write is attempted, which matters when the failing tool is not as well-behaved as customer.update was. A tool that partially applies a write before rejecting it, or that has a side effect on a rejected call, leaves the run in a state the plan cannot describe; a precondition means that tool is never reached.

The cost is per-run and the benefit is per-incident, which is a ratio worth computing rather than intuiting. Six steps with one probe each, at a measured 60–120 ms per probe against the billing and CRM APIs, add roughly 540 milliseconds to a run whose median wall time is 3.2 seconds — a 17% increase. Multiply the per-run cost by the queue's volume, about 900 accounts a month, and set it against the events it catches, which is a staleness firing on perhaps one or two of them. Every run pays; one or two benefit.

Whether that is a good trade turns on one variable, and naming it is more useful than the arithmetic. The benefit of a precondition scales with the size of the window between planning and execution, because that window is the only interval in which the world can drift from what the planner assumed. Its cost does not scale with anything — the probes take the same time whatever the window is. In this batch design the window is seven hours and forty-one minutes, which is a long time for the world to stay as the planner imagined it, and that is the strongest argument for paying. In an interactive variant of the same agent, where the plan is written 400 milliseconds before it executes, the same probes cost the same 540 milliseconds. The guard is then more expensive than the interval it guards, and the interval it guards is very nearly empty.

Preconditions also do not close the window between the check and the call, which is the ordinary time-of-check-to-time-of-use race, where the world changes in the gap between the probe and the call that relies on it. The window shrinks from hours to milliseconds and does not disappear, so a precondition converts a likely staleness into an unlikely one rather than into an impossible one, and any step whose correctness requires the check and the call to be atomic needs a mechanism the plan format does not provide.

The more important limit is the one that answers the proposal directly. A precondition that fails leaves you exactly where a postcondition that fails leaves you: holding a step that cannot proceed and a remainder written under an assumption that no longer holds. The response is still to void the remainder and re-plan with completed and remaining work in hand. So preconditions change the moment of detection from after the write to before it, and change nothing about what happens next — which means a system that adds them without building the recovery path has bought an earlier way to reach the same undefined behaviour, at 17% on every run.

The re-plan in this chapter was straightforward because customer.update is safe to attempt again — the 409 means nothing happened. The retry question gets harder when the failure arrives after the request has left the process, and the executor has no idea whether the operation took effect.