A plan followed without re-validation is a chain written at runtime

An executor, in a plan-then-execute agent, is the code that takes the list of steps the model wrote and calls the tools it names. The simplest one is a for loop over the list. That loop is where a reconciliation agent at a payments company wrote a five-page report, correctly formatted and entirely wrong, out of a file that had been deleted while the run was already under way — and did it without a single exception being raised or a single step being marked failed.

The interesting part is not that a model got something wrong. The plan was right. It named the correct file in the correct bucket, and when the checker from chapter 2 validated it, the file was there. What went wrong is that the world changed after the plan was written, the run observed the change, and nothing in the system was in a position to act on the observation. This chapter is about what an architecture looks like when the only component capable of reacting to information is never shown any, and about which of the properties that make ordinary control flow safe survive that arrangement.

The nine lines that turn an observation into a log line#

The March reconciliation ran at 09:02 on a Saturday. The plan was checked and clean; the export was in the bucket when the checker looked. At 09:11, while step 2 was still creating the staging table, a lifecycle rule on s3://vendor-drops/ with a 24-hour expiry deleted vendor/exports/2026-03-invoice.csv. The rule had been added in January and had never mattered, because until this month the reconciliation always ran within a few hours of the drop.

Here is the transcript from step 3 onward.

09:12:04  step 3  s3.get key=vendor/exports/2026-03-invoice.csv
          <- {"error": "NoSuchKey", "status": 404}
09:12:04  step 4  warehouse.load target=staging.vendor_invoice_202603 source=<empty>
          <- {"status": "ok", "rows_loaded": 0, "duration_ms": 210}
09:12:07  step 5  run_sql file=sql/reconcile_march.sql
          <- {"status": "ok", "rows": 61400}
09:12:09  step 6  write_report path=reports/2026-03-recon.md
          <- {"status": "ok"}
09:12:11  step 7  issue_credit source=step_5.overcharges cap_usd=500
          <- HELD: 61,400 credits exceeds approval threshold

The approval gate from chapter 2 is the only thing that stopped this, and it stopped it at the last possible instant, on a threshold that happened to be set low enough. Everything before that ran green. Step 4 reports success because loading zero rows is a legal load. Step 5 reports success because a reconciliation that finds every internal charge unmatched by a vendor line is a perfectly well-formed reconciliation; the SQL cannot tell the difference between "the vendor did not bill for these" and "there is no vendor data". The report step wrote a document, five pages long, correctly formatted, entirely wrong.

Step 5's result deserves more than the word "success", because the number it returned is the tell and it can be derived rather than asserted. A reconciliation compares two sides: the internal ledger's March charges, and the vendor's March invoice lines. To find charges the vendor never billed for, it has to keep ledger rows that match nothing on the vendor side, which is what an outer join does and an inner join does not. With the staging table empty, no ledger row matches anything, so every ledger row in scope survives the join as an unmatched discrepancy. The count of discrepancies therefore equals the count of ledger rows in scope, and nothing about the vendor side enters the calculation at all.

That is a closing identity, and what it licenses is narrow. rows: 61400 certifies that the join ran to completion over every March ledger row. It certifies nothing whatever about whether the vendor side had any rows in it, because the vendor side is a term that never entered the expression. A result that equals the size of one input, rather than being a function of both, is the general signature of an input that dropped out. Downstream, write_report and issue_credit receive a number that is indistinguishable from a legitimate one — and worse than indistinguishable, because a correct run would have had the vendor invoice's own line count on the other side of the comparison, and those two counts are what a healthy March looks like when they agree.

The uncomfortable half of that is the direction it runs in. An inner join would have returned zero discrepancies and issued no credits, so the failure would have been silent in the harmless direction. The reconciliation uses an outer join because an outer join is the only kind that can find a charge the vendor forgot to bill, which is most of the reason the job exists. The property that makes the query correct is the property that makes its failure expensive, and there is no version of the SQL that has one without the other.

The 404 was in the transcript at 09:12:04. Nothing read it.

def execute(plan, tools):
    transcript = []
    for step in plan["steps"]:
        result = tools[step["tool"]](**step["args"])
        transcript.append({"n": step["n"], "result": result})
        if result.get("error"):
            log.warning("step %s returned an error: %s", step["n"], result["error"])
    return transcript

The error is logged. The loop advances. This code is not a straw man — it is what a plan-execute executor looks like when somebody writes one in an afternoon, and the log.warning is what makes it feel handled.

It is worth being exact about what "nothing read it" means, because the code plainly does read it. result.get("error") is evaluated on every iteration. The value reaches a log line. What it never reaches is a branch. The only thing that decides which tool gets called next is the loop variable, and the loop variable is a function of plan["steps"] and nothing else. So the sequence of calls this executor will make is fully determined the moment the plan exists, and you could print it before the first request leaves the process, for any world at all.

That is the same shape as the dispatch receipt in the previous chapter, in its strongest form. The quantity that decides what happens next — the position in the step list — is constant while the dangerous property, whether the world still matches the plan, varies freely. A defence computed from inputs that exclude the thing being defended against cannot fire, and no amount of tuning it will help. The reason this one is easy to miss is that the error is named in the code, so the vocabulary of error handling is present while the control flow of error handling is absent, and a reviewer scanning for the word finds it.

The harness deliberately returns tool errors as data rather than raising, because raising on every 404 would make an interleaved agent unable to recover from anything, and that design decision is correct for the loop in topic 10.3, where the model sees the 404 on its next turn and decides what to do about it. Ported into an executor that iterates over a list, the same decision means errors are values that nothing branches on.

Two things are being distinguished there, and keeping them apart is most of the lesson. The requirement is that an agent must be able to carry on after a 404, because a 404 is often information rather than a catastrophe. The design decision — return errors as data instead of raising — is one way to meet that requirement, and it works by putting the error somewhere a decision-maker will read it. Under the interleaved loop the decision-maker is the model, consulted on every turn, so the error lands in front of it. Under a list-walking executor nothing is consulted between steps, so the same value lands in front of nobody. The decision has not changed; what changed is whether anything downstream reads it. A design decision whose value depends on a property of its caller flips sign when the caller is replaced, and that is why this one arrived correct and became a defect without anybody editing it.

Notice which piece of the previous chapter would have caught it. Step 3's success criterion was "the object exists and is non-empty" — a criterion the plan carried, in a field, that this executor never evaluates. The dependency edge from step 4 to step 3 was in the plan too. The plan was fine. The plan was, in fact, unusually well specified. What was missing was any code that consulted it after execution began.

A shell script with no if statements#

Strip away that this was produced by a model and look at what ran. A fixed sequence of seven operations, with a fixed order, no branches, and no conditionals, executed to completion regardless of what any operation returned. That is a shell script. It is a shell script with set +e and no if statements, which is a thing every backend engineer has written at some point and then learned not to.

Seven is worth checking rather than accepting, because it is the sort of number that gets written from memory. Chapter 1's plan for this task has five steps, running from fetching the export to issuing the credits, and the transcript above numbers those same five as steps 3 through 7 — which means two earlier steps exist, and the narration names one of them, the staging table creation that was still running at 09:11. Five and two is seven. The next chapter reaches the same total by an entirely different route, counting probes attached to steps, and lands on seven steps carrying eleven probes. Two independent counts agreeing is worth more than either on its own.

The difference is where the script came from. A shell script lives in a file. It has a path, a git history, an author, a review, and — if it does anything important — a test that runs it against a fixture. When it goes wrong you open it, find the missing if, add the if, and every future run has the if. The plan that ran at 09:02 was written at 09:02, executed once, and is now a JSON blob in a trace. Tomorrow's run gets a different one. There is no file to fix.

This is the trade being made, and it is worth stating without softening: model-authored plans give you the rigidity of a fixed control flow together with none of the properties that make fixed control flow safe. Rigidity is the part people notice — the executor marched past a 404 because marching is all it does. The missing properties are the part that costs more over a year. You cannot unit-test the graph, because the graph is different every run. You cannot code-review it, because it is authored 40 milliseconds before it executes. You cannot diff this run's control flow against last week's, because "control flow" is not a versioned artifact in this system. And you cannot fix a bug in it, because the bug lives in a distribution over plans rather than in a line of code, the planner having sampled this plan from many it could have written and being free to write a different one for the same request tomorrow — the best you can do is edit the prompt and hope the distribution moves.

The last of those claims needs its conditions attached, because unconditionally it is false and the rest of the chapter depends on it being false. It holds for bugs whose locus is the plan: a step the model tends to omit, an argument it tends to guess, an ordering it tends to flatten. It does not hold for bugs whose locus is the harness, and the harness is code with a file and a test like any other. The whole argument of this chapter is that the second category is the one you want your bugs in, and that moving a bug from the first category to the second is a repair even though the model still emits the same plans afterwards.

The 40 milliseconds is also a figure that rewards being read at the right rank. It is the gap between the plan being finished and the first tool call, and it is short because there is nothing between them. The window before the step that spends money is a different quantity, and it is as long as every step before issue_credit takes to run. That is precisely why chapter 1's plan puts issue_credit last: the review window a plan buys is measured from the plan to the first irreversible action, not from the plan to the first action of any kind. Read at rank one, the artifact looks unreviewable. Read at the rank that matters, it is the only reviewable thing in the run.

The tempting reply is that the plan is tailored: it was written for this specific task, with this month's dates and this bucket's key, which a generic workflow could never be. That is true, and tailoring is precisely the property that removes testability. A control flow that is identical across runs can be tested once and trusted thereafter. A control flow that is regenerated per request has to be validated per request, or it is not validated at all. Tailoring does not excuse you from the check; it is the reason you need one every single time.

Four architectures, same task#

Read that table by its gaps rather than by which column looks best. Set the literal-executor column beside the interleaved column and three rows are identical: neither is testable across runs, neither gives the same path for the same input, and both handle the long tail. The literal executor's entire advantage sits in the first three rows — there is a control-flow artifact, it exists before anything happens, and somebody can read it once per run. Its entire cost sits in one row, the fifth. That is the trade the previous chapters were built to exploit and the trade this chapter's incident collected on.

The row about the same input giving the same path is the one that makes the chapter's title exact. Inside a single run the literal executor is perfectly deterministic: given the plan, the sequence of calls follows with no freedom at all. Across runs it is not deterministic at any level, because the plan is drawn afresh each time. The rigidity and the variability are not in tension, because they are properties of different scopes. A run is a chain, and the chain is written at runtime.

One cell in the table is split rather than answered, and it is the design the rest of this topic argues for. Under a re-validating executor, "testable across runs" reads the checker is testable; the plan is not. Every other cell states a property of a whole architecture. That one states that a property has been separated into a part that varies per request and a part that does not, and then tested on the part that does not.

The last column is the uncomfortable one for anyone who reached for plan-execute in order to be more careful. The plain interleaved loop, with no plan and no ceremony, handles this specific failure better than the planned version does — the 404 is in the message array when the model chooses turn 4, so the agent is at least aware. It might still do something foolish with that awareness. It will not do the one thing the executor did, which is generate the observation and then discard it.

That is the sharp edge of the whole architecture. Plan-then-execute with a literal executor is the only one of these four designs in which the system observes a fact and then makes its next decision without reference to it. The plan is decided before observations exist; the executor does not reason; the model, which is the only component capable of reacting, is not consulted between steps. There is no seat at the table for information that arrives mid-run.

Where hand-authored workflows lose, and why any of this is worth doing#

None of the above is an argument that everything should be a coded workflow. The reconciliation pipeline should be — it runs monthly, its shape is stable, and it moves money, which is three good reasons to spend a week on a tested DAG and stop asking a model to reinvent it. The row-count investigation from chapter 1 should not be, and could not be: its second step depends on its first step's result across a branching factor no engineer wants to enumerate, and a workflow covering the ten most common causes of a row-count drop will be wrong on the eleventh in a way that a general agent would have handled.

The two examples are chosen to sit at opposite ends of one axis, and it is worth naming the axis rather than leaving it as a pair of anecdotes. What makes the reconciliation codeable is that its steps are known before any of them runs, so an engineer can enumerate the paths once and cover them. What makes the investigation uncodeable is that its second step is a function of its first step's result, so the set of paths is not enumerable in advance by anybody, engineer or model. A task's position on that axis is a fact about the task, and neither a model nor a better framework moves it.

The honest position is that a runtime plan is what you reach for when the task is too varied to code and too consequential to improvise, and that position is a real one that a lot of production agents occupy. What does not follow is that you can occupy it with a for loop. The properties a hand-authored workflow gets from being code — reviewed, tested, stable — have to be replaced by properties you build at runtime, and there are exactly two of them: a check before execution, which chapter 2 covered, and a check between steps, which nothing so far has.

The second one is where the 404 gets caught, and it is more work than it looks. It has to decide what a precondition even is for a step whose tool call has not been made yet. It has to get its facts from somewhere, and the obvious somewhere — the plan's own assumptions, restated — is the one place that guarantees the check agrees with the plan. And when a check fails it has to do something, and "log a warning and continue" is already taken.

The fault in the Saturday run was never the plan and never the model. A plan is a set of claims about a world, made at one instant, and claims made at one instant go out of date; that is a property of plans and not a defect in this one. The defect is architectural, and it is that the run acquired the information which falsified the claim and had nowhere to put it. Everything the executor did afterwards was correct given what it was allowed to know, which is the most expensive sentence in the whole incident.

What the plan bought, and it is real, is an artifact somebody could have read before the money moved. What it did not buy is any relationship between that artifact and the world it describes, once execution starts. Those are separable, and the separation is the design: the plan varies per request and carries the task's specifics, the harness stays still and carries the guarantees, and a check that runs between the two is the only place where a fact discovered at 09:12:04 can change what happens at 09:12:07.