Enumerable steps, not the presence of a model, decide it

Two arrangements are on the table whenever a design review argues about this word, and both are worth naming plainly before the argument goes any further. A workflow is a program whose steps, and the order those steps run in, are written down in code before anything executes: a fixed graph of nodes joined by edges, where any node may call a large language model (LLM) and the edges are still whatever the author typed. An agent, in the sense topic 10.1 gave the word, hands that arrangement over, because the model is given a set of tools and chooses at each turn which one to call, so the sequence of steps for a particular run exists only once the run is over. What separates the two is control flow, the part of a program that decides what runs next, which in the first case belongs to the person who wrote the code and in the second belongs to the model.

The pairing exists because a language model answers one prompt at a time and almost no real task fits in one prompt. Topic 4.6 covered splitting a task into steps a model can handle, and what remains after the split is the question of who decides which step runs next. A workflow pays for its answer at design time, in the effort of enumerating the steps, and repays that with runs that are testable, priceable and identical on identical input. An agent pays at runtime, in tokens, latency and variability, and repays that on work whose steps nobody could have listed in advance.

Both are already familiar under other names. The workflow is the nightly batch job, the document-ingestion pipeline, a ticket router, the Airflow DAG — a directed acyclic graph of tasks, which is a workflow under a scheduler's name for it — or the Step Functions state machine. The agent is the coding assistant that reads one file, decides from what it found there to read another, and stops when it judges the task done.

The objection that keeps these two apart from each other is reasonable on its face, and it is best met with a concrete design. Take a support-ticket triage service that takes roughly 4,000 tickets a day, whose first node is a model call that reads the ticket text and returns one of six labels, and where each label has its own downstream handler, written months ago, each with its own tests. The model reads the ticket, and what runs next depends on what the model said, so in an ordinary sense the model is choosing what happens next. Topic 10.1 drew the line at runtime control flow — a chain's edges exist before anyone runs it, an agent lets the model pick among actions each iteration — and this design seems to sit uncomfortably on that line, because the edge taken really is selected at runtime by a model.

The resolution is to make the criterion sharper: what decides the question is whether the steps and their order can be enumerated in advance for nearly every input. Not whether they are the same for every input, which would rule out any branch at all, and not whether a model is involved anywhere. Whether you can write the list down before the run.

Counting the reachable states#

For the triage service, the list is short enough to fit in a test file. Here is the whole of the control flow:

HANDLERS = {
    "billing_dispute": handle_billing_dispute,
    "password_reset":  handle_password_reset,
    "outage_report":   handle_outage_report,
    "feature_request": handle_feature_request,
    "abuse_report":    handle_abuse_report,
    "other":           handle_other,
}

def triage(ticket: Ticket) -> Resolution:
    label = classify(ticket)                      # exactly one model call
    handler = HANDLERS.get(label, handle_unrouted)
    return handler(ticket)

Seven paths exist through that function: one per key in HANDLERS, plus the one taken when the model returns a string that is not a key. Counting them is a matter of reading the code rather than of running it, which is the property being claimed. HANDLERS is a dictionary literal, so its keys are fixed when the module is imported; .get returns either the value for a key present in that dictionary or the default that was passed to it; and triage returns whatever the selected handler returns. There is one place where a choice is made and it has a bounded number of outcomes, so the number of paths is the number of keys plus one.

That seventh path matters more than it looks. HANDLERS is a closed dictionary built at import time, so the model's output is not a name that gets resolved against a registry, and it is not code that gets executed. It is a lookup key, and a key that misses falls into a branch someone wrote deliberately. If the model returns billing_disputes with an s, or a sentence of apology, or an empty string, the service does not enter an unspecified state; it runs handle_unrouted, which puts the ticket in a human queue and increments a counter.

The classifier's accuracy is a separate matter and a measurable one. On 1,200 hand-labelled tickets this one sits at 93.5% overall, with the confusion concentrated in a single pair — feature_request and other account for 61 of the 78 errors — and abuse_report, the branch where a mistake costs the most, has a recall of 98.1%, meaning it catches 98.1% of the tickets that genuinely belong to it. Those are the numbers the design review should have been arguing about, and they are properties of one node.

Two of those figures close on each other, and checking that they do is worth ninety seconds. An accuracy is a rate over a denominator, so the error count is the denominator multiplied by the complement of the rate. The denominator is twelve hundred hand-labelled tickets. The complement of the accuracy is six and a half tickets wrong in every hundred. Twelve hundred tickets at six and a half wrong per hundred is seventy-eight wrong tickets — which is the figure the sentence already gives, arrived at from the other direction. Nothing is left over, so there is no unstated set of failures hiding between the accuracy and the error count.

What that closure licenses is a plan rather than a reassurance. Sixty-one of the seventy-eight errors sit in one confusable pair, which leaves seventeen spread across every other combination of six labels — thin enough that no other pair can be a second cluster of any size. Accuracy work on this classifier therefore has exactly one target, and the size of the prize is known before anyone starts: eliminating that pair's confusion moves the headline rate by a predictable amount, because the remaining errors cannot be more numerous than seventeen.

There is a residual in that reporting worth naming rather than passing over. The class where a mistake costs the most is described by its recall — the share of tickets that genuinely belong to it which the classifier catches — and no precision is given, meaning the share of tickets routed to it that genuinely belong. Those two answer different questions. Recall bounds how many abuse reports are missed; precision bounds how much of the trust queue's time is spent on tickets that were never abuse. A branch reported on one of the two is measured on half of the cost of getting it wrong.

The enumeration is not a rhetorical claim, and it is worth making it executable, because a system that claims a closed branch set should fail loudly the day someone opens it:

def test_reachable_paths_are_enumerated():
    expected = {
        "billing_dispute", "password_reset", "outage_report",
        "feature_request", "abuse_report", "other",
    }
    assert set(HANDLERS) == expected          # no branch added without a test change
    assert triage.__code__.co_names.count("HANDLERS") == 1
    for label in expected | {"nonsense", "", "billing_disputes"}:
        with stub_classifier(returns=label):
            assert triage(SAMPLE_TICKET).status in TERMINAL_STATUSES

Nine cases, run in about a second against a stubbed classifier — a stand-in that returns whichever label the test asks for, so the branching is exercised without a model call — and they cover every state the service can reach. That is the operational meaning of enumerable. Every input the system will ever see lands in one of seven places, each of which has a test, an owner, a latency profile and a defined failure mode. The model influenced which of the seven, and it never held control flow, because control flow was decided when someone typed the dictionary.

It is worth separating what is required here from what was decided. The requirement is that tickets reach the handler that can resolve them. Six labels rather than nine, a dictionary literal rather than a registry, and a named default rather than an exception are three design decisions taken in service of that requirement, and each could have gone the other way without the requirement changing. Confusing the two is how a property of one afternoon's code becomes a claim about what the domain permits.

A lookup key is not a name that gets resolved#

The tempting mistake here is a belief that the model has taken control flow, on the grounds that the model picked the edge. It is tempting because the two situations are indistinguishable from where most people are standing: a model emits a string, and moments later a function runs that did not run last time. The observable is identical. The property that separates them is invisible at the call site, which is why the argument in a review tends to be conducted on the observable.

What is actually true is that the model's output is a lookup key. It is compared against a set of keys fixed when the module was imported, and a key that is not in the set selects a value chosen by the author rather than by the caller. The model never names something that then gets found; it names something that either was already there or was not. Nothing about the string it produced can add a member to the set, so the set of things that can run is the same on every run of the program, whatever the model says.

The quantity being held constant across these two cases is that one model call selects one branch, and that quantity is the same in both. The property that actually varies, and that decides whether anything is bounded, is whether the set of destinations is closed. HANDLERS.get(label, handle_unrouted) fails closed, because a miss is a branch someone wrote. A resolver that looks the string up in a registry other teams can register into fails open, because a miss can become a hit after a deployment nobody on this team made. Reading only the constant — one model call, one branch — tells you nothing about which of those two you are holding, and it is the one people read.

The slack in "nearly every input"#

The criterion says the steps can be listed for nearly every input, and the qualifier is not a hedge to make the rule easier to satisfy. It is there because a closed branch set applied to an open world will always meet inputs it has no branch for, and the design question is what happens to them. A system that enumerates six branches and crashes on the seventh case has not enumerated anything useful; a system that enumerates six branches and routes everything else into a seventh, deliberately built, counted branch has a total function over its input space.

That is what makes the seventh path in HANDLERS.get(label, handle_unrouted) structural rather than defensive. It converts "we did not anticipate this" from an exception into a state with an owner, and it gives you a rate — the proportion of tickets landing in handle_unrouted, which on this service runs at 0.4%, or about 16 a day. Sixteen tickets a day going to a human is a cost somebody can price, and it is visible in a dashboard rather than in a stack trace.

That daily figure is a second closure, and it is the product of exactly two quantities with nothing else in it: a rate of four tickets in every thousand, and a volume of roughly four thousand tickets a day. Four in every thousand, over four thousand tickets, is sixteen. Since the rate and the volume account for the whole of it, there is no third factor — no seasonal effect, no subset of ticket types — hiding inside the number waiting to be discovered; anything that moves it has to move one of the two.

What that licenses is the end of a certain kind of argument. Two people can disagree about whether a system's steps are enumerable, and they cannot disagree about whether 0.4% or 40% of production traffic is falling through, because it is a counter. When the unrouted rate is under a percent, the enumeration is a good description of the domain. When it climbs, the enumeration has stopped describing the work, and that is information about the domain rather than about the architecture.

The residual in that arrangement is that the "nearly" is bounded by this rate and by nothing else. There is no independent check that the six labels carve the domain at its joints — a ticket routed confidently to the wrong one of the six never touches the counter at all, and the unrouted rate will sit happily under a percent while it happens. The counter bounds how much the enumeration fails to cover; it says nothing about how well it covers what it claims.

There is a related question about what counts as a step, and the honest answer is that granularity is chosen by the designer. handle_billing_dispute is one step at the level of this diagram and eleven functions at the level below it. The criterion applies at whatever level the control flow is being argued about, and the useful level is the one where the branching happens, because that is where the state space either closes or does not.

Where routing turns into control flow#

The boundary is real, and it is close by. Three modifications to that same diagram move it across, and each one is a change a reasonable engineer might make in an afternoon without noticing what it costs.

The first is a back edge. Suppose that after handle_billing_dispute runs and finds no matching invoice, the result goes back to classify for a second opinion. The node set is still six, and the path set is no longer six, because a path is now a sequence and sequences compose. With back edges from every handler to the router, the reachable paths of length three number 216, and there is no length at which they stop, so the only bound on the run is a counter someone remembered to add. Nothing about that change looks dramatic in a diff. It converts a system whose behaviour you can list into one whose behaviour you can only sample.

One relation is doing all of that work, and having it explicitly is what lets you predict the next case instead of recomputing it. Each time control returns to the router it faces the same six choices, and the choices at one visit do not restrict the choices at the next, so the count multiplies by six per visit. One visit gives six paths. Two visits give six times six, which is thirty-six. Three visits give six times thirty-six, which is the figure quoted above. The general form is six multiplied by itself once per visit, and what it licenses is a prediction at any depth without drawing another diagram: a run allowed ten visits has a path count with eight digits in it, and the reason nobody enumerates that is arithmetic rather than taste. The same relation is why the bound has to be a counter — the count grows without limit as the number of visits does, so no finite list of paths exists to be written down.

The second is an open action set. If the handler is chosen by resolving the model's string against a plugin registry that other teams can register into, then the set of things that can run is no longer knowable from this file. HANDLERS.get(label, handle_unrouted) fails closed; registry.resolve(label) fails open, and the difference is the entire question.

The third is feeding a handler's output back into the model's choice. Once the model sees the result of the refund lookup before deciding what happens next, later steps depend on what earlier steps discovered, and the designer cannot write the list because part of the list is produced during the run.

A fourth case comes up often enough to be worth settling explicitly, because it looks like it should cross the line and does not. Suppose handle_billing_dispute begins with its own model call that sorts the ticket into four sub-cases — duplicate charge, wrong currency, disputed refund, cancelled subscription — each with its own handler. There are now two model calls making two routing decisions, and the reachable state space is 6 × 4 for that branch and 1 for each of the others, which is 24 plus 5 plus the unrouted paths. Still finite, still writable in a test, still one path per run with no back edges. Nesting routers multiplies the state count and does not change its character, and the practical limit is not the criterion but readability: at four levels of nesting the enumeration exists and nobody can hold it, which is a maintenance problem rather than a control-flow one.

Counting any nesting takes one rule, and it is worth having because the rule, and not any particular total, is what settles the next case. Each top-level branch contributes the number of paths that exist below it, and a branch with no router beneath it contributes one; the total is the sum of those contributions, plus the default branch. Multiplication enters only inside a branch that has a sub-router, where its own outcomes multiply against that one branch and against nothing else. Multiplying the two routers' widths together instead would count combinations no run can reach — a password-reset ticket arriving at a wrong-currency handler — which is the thing to watch for, since the finiteness the paragraph above depends on holds either way and the count does not.

That second diagram is an agent by 10.1's definition and by this chapter's, and the interesting part is how little visual difference the back edges make. The distinguishing question to ask of any design is not how many components it has. It is: can I write down, before the run, the set of states this can be in, and is that set finite and small enough to test?

Read that table across rather than down. Each row is a gap between two designs on one question, and the gaps are not the same size. The first two rows are the criterion restated, and a team can argue about them. The last three are where the gap turns into work somebody has to do on a Thursday: a repro that exists or does not, a failing case that is one stored input or a trajectory that may never come back, and a change that is a dictionary entry against a change that is a prompt edit plus a re-evaluation. A design review that argues the first two rows and skips the last three has argued about the vocabulary and left the cost unexamined.

Stakes, cost, and the library you happened to import#

The criterion does not say the routing decision is low-stakes. A ticket misrouted from abuse_report to other is a real failure with a real person on the end of it, and the classifier's accuracy is exactly as load-bearing as it would be inside an agent. The claim is narrower: that failure is contained inside a branch someone wrote, reproduces from a single stored input, and shows up as one number — the confusion matrix of a six-way classifier — rather than as a distribution over paths.

It also does not say the fixed version is always cheaper to run. This service makes one model call per ticket at about 900 input tokens, so its cost is flat and its p99 — the latency that 99% of requests come in under, the notation used throughout this topic for a tail rather than an average — is the classifier's p99 plus the handler's. A version that made zero model calls and used a keyword rule would be cheaper still, and would be worse, and the criterion has nothing to say about that trade. The criterion answers one question — whether you have handed over control flow — and answering it does not settle whether the design is good.

And it does not depend on which library you used. An agent framework configured with six tools, a single-iteration cap, and a forced tool choice compiles down to exactly the router above; the reachable state space is six, and calling it an agent because of the import statement would be a claim about the dependency list. Equally, a hand-rolled while loop with no framework anywhere in it, where the model picks the next action each turn from an open set, has handed over control flow completely. What is in the state space is decided by the shape of the code, and the vocabulary follows the shape rather than the other way round.

Shipping without the trajectory store was the right call on the evidence available, and it is worth being precise about what that evidence was: a branch set that could be listed, a default branch that made the listing total, and a counter showing how often the listing failed to cover an input. Those three are what the criterion asks for, and any design that has them can be argued about with numbers instead of with vocabulary.

It is not, however, an argument that generalises, and the sentence that most often follows it in a design review is about resilience — that the system should be able to retry when a downstream API fails, and that retrying is the kind of behaviour an agent gives you. That sentence is doing something quite different from the one this chapter answered, and it needs taking apart before it costs anyone three weeks.