An exit the model controls is not a termination condition

Every agent is a loop, and a loop needs a reason to stop. The reason usually offered is that the model will answer eventually instead of asking for another tool, and that is an observation about how the model has behaved so far rather than a property of the program. The two are different kinds of statement, and telling them apart is the whole of this chapter. A termination condition is a claim about your code that holds whatever the input turns out to be: after so many passes, or after so many seconds of wall clock, control leaves the loop. An exit is anything that in fact removes control from it, which includes a gateway timeout, an exception nobody caught, and the model deciding it is finished. Every termination condition is an exit. Most exits are not termination conditions.

Getting that backwards costs money, and it costs it in a particular direction. Each pass re-sends the whole conversation, so the input side of the twentieth request carries everything the nineteen before it produced. An agent that fails to stop therefore does not spend a little more than one that stops; it spends on a curve, while holding a worker and a user's connection open. The failure in the other direction costs nothing at all, which is why it survives longer: a loop that leaves too early returns quickly, and quick looks like health.

A working tool, and a loop with nothing left to stop it#

The runaway that matters is not the one where something breaks. Here is a trace from a code-review agent pointed at a repository that had been archived and renamed, with the model asked to find a class that was no longer there:

iter  1  grep_repo("class BillingAdapter")        -> 0 matches
iter  2  grep_repo("BillingAdapter")              -> 0 matches
iter  3  grep_repo("billing_adapter")             -> 0 matches
iter  8  grep_repo("class .*Billing.*Adapter")    -> 0 matches
iter 19  grep_repo("Adapter")                     -> 0 matches
iter 34  grep_repo("billing")                     -> 0 matches
iter 51  grep_repo("BillingAdapter|BillingClient")-> 0 matches

The chain that produced that has five links, and each one is checkable on its own. The repository the tool was pointed at had been archived. An archived repository still answers a search, so the tool was returning an empty result set rather than an error. An empty result set is a valid answer, so nothing in the loop ever looked wrong from the inside — the model had a question, a tool that answered questions, and answers that kept coming back empty. An empty answer to a specific query is a defensible reason to try a less specific one, so every one of those calls was a defensible next move given the previous one. And the only exit this loop had was the model deciding it was finished, so removing the model's reason to finish removed the loop's only way out. Nothing malfunctioned at any link in that chain, which is the property to carry forward: a tool working correctly can delete a loop's only exit.

The trace is a sample rather than a record, and it is worth naming that before anything is built on it. Seven rows are shown out of fifty-one calls — iterations one, two, three, eight, nineteen, thirty-four and fifty-one — and the forty-four in between are not. Every claim in the paragraph above is therefore a claim about the sampled rows: that each call was a defensible next move, that the result set was always empty, that the model never repeated a query exactly. The sample supports those claims and does not establish them, and a reviewer who wanted them established would ask for the whole trace.

What that run cost can be read off two numbers without instrumenting anything, and the reading is worth doing because it generalises to any loop. Call the number of model requests in a turn the request count, and call the size of the conversation on the last request the final context. Every request re-sends everything that came before it, so the first request carries almost nothing, the last carries the whole final context, and the sizes in between rise fairly evenly. The average request therefore carries about half the final context, which makes the cumulative input for the turn roughly the request count times half the final context.

Turn that relation around and the archived-repository run gives up its shape. Fifty-one model requests against about six million cumulative input tokens means the average request carried a little under a hundred and twenty thousand tokens, and the final context was about twice the average — of the order of a quarter of a million tokens on the last request alone, for one pull request comment. Nobody measured that number; it falls out of two figures any trace already contains. What the relation licenses is pricing a runaway before you have one: take the cap you are considering, take the context a turn of that length would reach, multiply, halve, and multiply by the input rate. It is an order-of-magnitude reading rather than an exact one, because it assumes the conversation grows at a steady rate, and a turn whose tool results arrive in wildly uneven sizes will sit above or below it.

Every path out, and who owns it#

The productive exercise is not to argue about whether the model stops. It is to write down every way control leaves the loop, and beside each one, who decides. Here is an agent loop as a state machine with the exits drawn:

Four arrows leave the machine and they are not equivalent. One belongs to the model: the turn that comes back with no tool-use block, and it is the one everybody names first. One is yours: the iteration cap. One is an accident: an exception escaping dispatch. The fourth is drawn as a single arrow labelled provider error or timeout, and it is really two things — a provider that failed after your retries were spent, and a request timeout imposed from outside your process. Splitting that arrow, and adding the wall-clock deadline a loop can carry although this one does not, gives six ways out, and the same question has to be asked of each.

The right-hand column is best read as a gap rather than as a list of levels. Two rows answer yes and four do not, and the four fail for three separate reasons: the model's exit is not yours to constrain, the two accidental exits belong to nobody and fire only once something has already gone wrong, and the infrastructure exit fires at a boundary that knows nothing about the conversation it is ending. What the gap licenses is a review question with one correct answer. Ask which exit bounds this loop, and if the answer names a row from the four, the loop has no bound — a verdict available from the table alone, before anything is run and without reading a single trajectory.

Only two rows in that table are termination conditions in the sense of a property you can state and defend. The rest are events. The strongest objection to "it always stops because the model eventually stops calling tools" is exactly this: the only exit that sentence names is one your code does not control, so the claim reduces to a prediction about model behaviour on inputs you have not seen. The grep trace above is what that prediction looks like when it is wrong, and note that nothing malfunctioned to produce it — a tool returned a valid, empty answer, repeatedly.

The accidental exits deserve a moment even though nobody plans them, because an exception escaping dispatch is the one that does collateral damage. It leaves the loop between the two appends from chapter 2, so a conversation persisted at that moment has an assistant turn whose calls were never answered and cannot be resumed; it produces a 500 rather than an answer the user could have used; and it is indistinguishable, in most dashboards, from the loop having a bug. Catching tool failures at the dispatcher boundary and turning them into error results is what converts that exit into an ordinary iteration, which is the same fix as the second bug in chapter 2 seen from the outside.

Two other shapes remove the same exit. The first is the identical repeated call: the model asks for get_order("AB-40912"), reads a result that does not contain what it expected, and asks for get_order("AB-40912") again, byte for byte. This one is more unsettling than the reformulation case because it has no story — the model is not exploring, it is stuck — and it is common enough after a tool returns a large or confusingly shaped payload that a repeat detector in the dispatcher earns its keep. The second is subtler and involves a tool that is honestly doing its job: a wait_for_job tool that returns {"status": "running"} gives the model a perfectly good reason to call it again, and if the underlying job never finishes, the loop never does either. That last one is worth internalising, because the tool is not broken and the model is not confused; the design simply has no bound in it anywhere.

Those two and the archived repository are one shape wearing three costumes. In each case a tool returns something valid, the model reads it as a reason to continue, and the model-controlled exit recedes by one more step. Nothing in the system is in an error state at any point, so no error-driven defence fires, and that is why the bound has to be a count or a clock rather than a check for something going wrong.

The exit that fires when it should not#

There is a failure in the other direction, and it is the one that goes unnoticed for longest because it produces no incident, no latency and no bill. Most loops are written like this:

if response.stop_reason != "tool_use":
    return text_of(response)

That branch treats four different outcomes as one. end_turn means the model finished a complete answer. max_tokens means it was cut off mid-sentence — the text is a fragment, and if it was cut off while emitting a tool-use block, you get a truncated turn that may not even parse. A refusal means safety classifiers declined the request and the content is empty or partial. A pause_turn means a server-side tool ran out of its own iteration budget and the turn is meant to be resumed by sending it back. Three of those four are not the model saying it is done, and the branch above returns all four to the caller as an answer.

The stumble here is not a slip. It is a belief that a defence is working, and the defence is the dashboard: a team watching latency and completion rate believes it would notice a loop misbehaving, on the grounds that a misbehaving loop is slow and a broken one errors. Neither holds for this failure, and the metrics move in the reassuring direction while it spreads.

What that looks like in production is an agent that "answers" a question in 1.4 seconds with an empty string, or with two-thirds of a sentence, and a success metric that counts it as a completed turn. One team found theirs by noticing that their p50 latency, the time under which half of all turns finish, had improved week over week while their thumbs-down rate had doubled.

Those two movements are the same event counted twice, and the mechanism is worth walking one link at a time. A turn that exits on max_tokens or refusal makes no further tool calls, so it returns sooner than a turn that runs to a real answer. The loop returns it through the same branch as a real answer, so the metric records it as a completed turn. The median is computed over completed turns, so adding fast members to that population pulls the median down. Each premature exit therefore improves the latency figure and costs one user an answer, and the two counts have to move together because they are the same turns. The definition of a completed turn stayed constant while the reasons a turn could complete varied underneath it, which is precisely why a metric built on that definition could not see the change. The improving median was evidence of the fault, and reading it as reassurance is the whole of the mistake.

The mistake is tempting for a reason that has nothing to do with carelessness. stop_reason != "tool_use" reads in English as "the model is not asking for a tool, so it must be answering", and that reading is correct for one of the four values the field can hold. It also matches how the loop gets drawn on a whiteboard, where there are two cases: another tool, or done. What is actually true is that "not asking for a tool" and "finished" are different propositions, and the provider reports them with different values because they have different consequences for whoever called you. The fix is to branch on the value rather than on its inequality with one constant:

match response.stop_reason:
    case "tool_use":
        pass                                   # continue the loop
    case "end_turn":
        return Answer(text=text_of(response), status="complete")
    case "max_tokens":
        return Answer(text=text_of(response), status="truncated")
    case "pause_turn":
        messages.append({"role": "assistant", "content": response.content})
        continue                               # resume; counts against the cap
    case "refusal":
        return Answer(text="", status="refused", detail=response.stop_details)
    case other:
        return Answer(text="", status="unknown_stop_reason", detail=other)

The case other arm matters more than it looks. Stop reasons are an open set that providers add to, and a loop written as a two-way branch will silently reclassify a new one as a successful answer on the day it first appears.

Separating what is required here from what is chosen keeps a review on the right argument. The requirement comes from the system: the loop must terminate, and the caller must be able to tell what it got. Branching on the value of stop_reason rather than on its inequality with one constant is a design decision, and treating an unrecognised value as unknown rather than as success is a second one. Both decisions cost a few lines and both can be reversed; the requirement can be neither cheapened nor reversed. A review that has not made that separation argues about style when it should be asking whether the caller can distinguish a truncated answer from a complete one.

What the twelfth iteration returns#

Set the cap at twelve and the question becomes concrete: control reaches the bottom of the loop, the counter is exhausted, and something has to be handed back. Three answers are common and two of them are wrong.

Raising an exception is the most common and it discards work. By iteration twelve the conversation usually contains real findings — three files read, an order fetched, two charges identified — and the user's question may well be answerable from them. A RuntimeError throws that away, returns a 500 to a caller who now has nothing, and if you are persisting as you go, leaves a stored conversation in whatever state the last append put it in. Retrying the whole turn is worse: the same input produces the same trajectory often enough that you have doubled the cost of a failure and delayed it by another twelve iterations.

The version that holds up is a terminal result, in the same shape as a successful one, that says plainly which exit fired and carries the partial work:

return Answer(
    text=final_text,                     # from one last tool-free model call
    status="iteration_cap",              # names the exit, not "error"
    iterations=MAX_ITERATIONS,
    partial=summarise(messages),         # what was actually established
)

Three properties make that useful rather than decorative. It is the same type as the success path, so callers are not obliged to catch anything to handle it, and a caller that ignores status still gets text rather than a stack trace. It names the exit, which means your dashboards can count cap-trips as their own category instead of burying them in a 5xx rate that lumps every server-side failure together — and the number of cap-trips per thousand turns is the single most informative metric about whether the cap is set correctly. And it carries the partial work, so the user gets "I found two charges of £42.00 on AB-40912 and could not determine whether the second was refunded" rather than silence.

Getting real text out of that final position takes one more model request with tool calling switched off, so the model is forced to answer from what it has. That request costs a couple of seconds and turns a dead turn into a useful one, and it is the difference between a cap that protects your infrastructure and a cap that also protects the user's time.

That last request deserves a sentence of its own, because it looks like the loop refusing to stop after being told to. It is not another iteration. Tool calling is switched off, so the response cannot contain a tool-use block, so the response cannot cause another pass — the exit is guaranteed by the shape of the request rather than by the model's cooperation. That is the distinction the whole chapter turns on, applied one more time at the point where it is easiest to lose.

A counter cannot see progress#

A cap bounds cost, latency and context growth. It does not tell you whether the loop was making progress, and nothing in this chapter does. Iteration two and iteration nineteen are indistinguishable to a counter, even though one was worth running and the other was the fourth reformulation of a query against an archived repository. The signals that would separate them are all partial: hashing tool results catches exact repeats and misses the grep trace above, where every call was different; comparing successive results by embedding similarity, a numeric score for how alike two pieces of text are, produces false stops on searches that are legitimately narrowing; asking a second model whether the loop is converging costs a model call per iteration to save model calls. The state of the practice is a blunt number tuned from your own distribution of iteration counts, plus a repeat detector in the dispatcher, plus the discipline of returning something useful when the number trips.

The gap between those two things is worth stating plainly rather than leaving as a caveat. A bound is a claim about resources: this turn will not cost more than so much, take longer than so long, or grow its context past so many tokens. Progress is a claim about the work: this turn is closer to an answer than it was three iterations ago. No count of iterations can give you the second claim, because the counter has no access to what the iterations contained. Anything that does carry it needs to read the results, and every instrument that reads the results is either fooled by the reformulation case or fires on legitimate narrowing.

If you are asked which of these failures you have actually seen, the honest ordering is worth having ready, because it is not the ordering the list suggests. The premature exit is by far the most frequent and the least reported, since max_tokens truncation happens on any turn where the output budget is tight and nothing about it looks like an incident. Tools that return empty rather than erroring come next, and they cluster around configuration changes — a renamed index, a rotated credential that yields an empty result set instead of a 401, an archived repository. Exact repeated calls are third and cluster around large tool payloads. Genuinely unbounded loops that nothing stops are rarest, because most systems have an infrastructure timeout somewhere that eventually kills the worker, which is precisely why teams believe their loops terminate: something does stop them, and it is not the code they wrote.

The rule that comes out of all of this is short enough to apply in a review. Write down every way control can leave the loop, mark each one with who decides it, and if the marks say the model, an accident or infrastructure, there is no termination condition in that code — there is a habit that has held so far. Adding the counter is the easy half of the repair. The half that gets skipped is deciding what the pass after the last one hands back, and the version that survives contact with a caller is a result of the same type as a success, naming the exit that fired and carrying whatever the turn established. A cap that returns a stack trace protects your infrastructure from the loop and does nothing at all for the person who asked the question.

There is also a limit on what you can conclude from your own code. Everything here assumed you could see the loop, count the exits and change the branch. The harder version of the question arrives when the loop was written by somebody else, the tools point at systems you cannot call, and someone needs a decision by Thursday.