Retryable and terminal error classes

An error body written to name the violated precondition, echo the offending value and state the permitted set is a large improvement, and it still leaves one question unanswered: whether trying again could ever work. That question is about the future, and a message describing a state is about the present, so a caller reasoning from the message alone is being asked to infer a policy from a fact.

Here is what that looks like on a payments tool. An agent calls transfer_funds for 150.00 and gets back:

{"error": "insufficient_funds", "message": "Balance 42.10 below requested 150.00", "balance": 42.10}

That response has a stable code, a specific message, the offending value and the boundary it crossed. The agent retried the identical transfer immediately. What the response does not say anywhere is that repeating this call cannot work.

Why a clear rejection still reads as maybe#

"Balance 42.10 below requested 150.00" is true at 09:14:22 and says nothing about 09:14:24, and balances genuinely do change — a deposit clears, a hold releases, an earlier transfer in the same agent run settles. The inference the caller is being asked to make is not sound, because the fact is compatible with both outcomes.

Setting out what the caller is being asked to do makes the gap plain. The response states a fact about the present. What the caller needs is a claim about the future: whether this same call, made later, could succeed. Getting from one to the other requires two things the response does not contain — whether the quantity in the fact can change at all, and whether anything is likely to change it in the interval a retry would cover. A balance can change and frequently does. A per-transaction ceiling cannot, in the sense that nothing the agent or the customer will do in the next few seconds moves it. Yet both produce identically shaped bodies: a value, a threshold, and a comparison between them. The caller cannot tell them apart because the difference does not live in the fact. It lives in the policy behind the fact, and the response carries the fact.

The caller's prior — the behaviour it falls back on when the response gives it nothing specific to go on — pushes the same way. Failures a model has seen most often in text are transient ones dressed in exactly this register, and the accepted remedy in nearly all of that material is to wait and try again. Absent a signal to the contrary, retrying is the move with the highest expected value from the model's point of view, and it is the only move that costs nothing to attempt. An agent that stops on the first failure is worse on the whole distribution of failures, so the behaviour is not a defect in the model; it is the correct default applied without the one piece of information that would override it.

The information has to be explicit, machine-readable and separate from the prose, because prose is what the model already had. A boolean is enough to stop the hammering, and three classes are enough to also point at what to do instead.

The distinction between the second and third rows is what a plain retryable: false collapses. Both are permanent for the arguments as sent, and they differ in whether the agent's own next action can fix them: an over-limit amount is under the agent's control, a taken slot is not. Flattening them produces an agent that correctly stops retrying and then reports failure on a call it could have repaired, which is an improvement over hammering and is still a task that did not get done.

Two properties of that graph matter more than its shape. Every leaf is a distinct outcome, so an agent following it never reaches "the system is down" by exhaustion — that report is reserved for the transient branch running out of attempts, which is the one case where it is true. And the branch is selected by a field rather than by reading the message, which means the behaviour is stable when someone rewrites the copy.

The transient branch is also where the cost of routing a failure into the wrong one is easiest to compute, because exponential backoff makes the waiting explicit. Doubling waits of one, two, four and eight seconds add to fifteen seconds of deliberate delay, on top of a model round trip before each attempt, before the attempts are exhausted. Spent on a dependency that is genuinely flaky and does recover, that is fifteen seconds well spent and the branch ends at "continue task". Spent on a rejection that will be identical every time, it is fifteen seconds and four round trips buying nothing, and it ends at the leaf marked escalate — which on a misrouted call produces a report of an outage that is not occurring. The schedule is not the defect and removing it would break the case it was added for. Sending a terminal failure down it is the defect.

A terminal failure with somewhere to go#

Terminal failures divide again on whether the domain has a neighbouring option. Scheduling almost always does, since a taken slot implies a calendar that knows which slots are not taken.

{
  "error": "slot_unavailable",
  "class": "terminal",
  "retryable": false,
  "message": "Thursday 2026-03-12 14:30 with Dr Osei was booked 2 seconds ago and cannot be reserved.",
  "requested": {"practitioner_id": "dr_osei", "start": "2026-03-12T14:30:00Z"},
  "alternatives": [
    {"practitioner_id": "dr_osei", "start": "2026-03-12T14:45:00Z"},
    {"practitioner_id": "dr_osei", "start": "2026-03-12T16:00:00Z"},
    {"practitioner_id": "dr_yilmaz", "start": "2026-03-12T14:30:00Z"}
  ],
  "alternatives_truncated": true
}

The alternatives array converts a dead end into a choice the agent can put to the patient in one turn. It is capped at three deliberately, with alternatives_truncated telling the model that a search tool will find more, because an unbounded list turns a 60-token error into a 4,000-token calendar dump on a failure path that fires on perhaps 3% of bookings.

Both ends of that comparison are worth holding together, because the cap looks arbitrary until they are. Three alternatives is three short objects and keeps the whole body near 60 tokens. An uncapped list is however many free slots the practitioner has across the surrounding days, which for a working calendar is the 4,000-token figure and is a calendar rather than an error. Multiply either by the 3% of bookings that reach this path and the capped version disappears into the noise while the uncapped one does not. The cap also costs nothing in usefulness, because a patient is not choosing among a hundred slots — they are choosing among the two or three nearest to what they asked for, and the rest of the calendar would be read past. What alternatives_truncated buys is that the cut is declared rather than silent, which is the same obligation a capped page of rows had when it carried matched alongside returned.

The cost is a second query on the error path. In a clinic's case that is an indexed range scan over the practitioner's availability for the surrounding four hours, measured at 8 to 14 ms against a table of about 40,000 future slots, on a request that has already failed and is therefore not on the latency-critical path for a success. That is cheap enough to run unconditionally. It is not always cheap: an alternatives lookup that costs 400 ms, or that hits a third-party API with its own rate limit, is one you gate behind a flag or omit entirely, and omitting it is a legitimate design — class: terminal with no alternatives still stops the hammering, which was the expensive part.

Why the cheap version is acceptable at all deserves a sentence, because "a second query on the error path" sounds like a plain regression. A booking that succeeds never runs the lookup, so nothing on the path a patient normally takes gets slower. A booking that has failed has already spent one model round trip and is about to spend another on whatever the agent does next, and 8 to 14 ms set against a round trip measured in seconds is not observable. The trade turns only when the lookup stops being cheap or stops being local. A 400 ms call is paid on every failure, and a call into a third party with its own rate limit is paid out of a budget that a burst of failures is already consuming — which is how an alternatives lookup added to help turns one outage into two.

The insufficient-funds case has no adjacent value to offer, since 42.10 is not an alternative to 150.00 in any sense the agent should act on unilaterally. What it can carry is the tool that changes the blocking condition, where one exists and the agent is permitted to use it — {"blocked_by": "balance", "unblock_with": "request_deposit"} — and where none exists, the honest terminal response with the two numbers, which lets the agent tell the user something specific instead of something alarming.

Two details of placement decide whether any of this survives contact with a real tool surface. The class field belongs at the top level of every error body, in the same position under every tool, because a model that has learned the shape from one tool applies it to the next one without being told; burying it under a meta object on half the tools produces an agent that reads it sometimes. And retry_after needs to be a number of seconds rather than a timestamp or a duration string, since a model asked to compare 2026-03-12T14:30:04Z against a clock it does not have will either guess at the interval or ignore the field. A transient branch that returns {"class": "transient", "retry_after": 2} and nothing else is enough for the harness to sleep and the model to describe the delay honestly if it has to surface one.

Status codes, harness tables and richer taxonomies#

HTTP status codes are the obvious existing vocabulary and they nearly work. 5xx is broadly transient, 4xx broadly not, and 429 carries Retry-After already. The failure is at the boundaries that matter most: 409 Conflict covers both the slot taken by someone else and the optimistic-locking conflict, where two writers raced on the same row and the loser is expected to re-read and try again. 400 covers a malformed enum, which the agent can repair, and a business rule that will never permit this operation, which it cannot. Codes describe what the protocol saw; the retry class describes what the caller should do next, and those two coincide only some of the time. Where the status is unambiguous it is worth setting correctly for every other consumer of the endpoint, and it is not the field the agent branches on.

None of that is a design fault in HTTP, and treating it as one leads people to try to fix it in the wrong place. A status code names what the protocol observed, and a three-digit space shared by every consumer of every endpoint has no room to also name what one particular kind of caller should do next. Splitting 409 into two codes would change the meaning of a code that existing clients already branch on. Adding a header to carry the distinction is inventing a field, and once a field is being invented it may as well sit in the body, where the caller that needs it is already reading. So the conclusion is not to abandon status codes but to stop asking them for something they do not encode: set the status correctly for the consumers that use it, and put the retry class where the caller who has to act on it will look.

A second option is to say nothing in the response and encode retry policy in the harness, keyed by tool name and error code in a lookup table. That works and is how a lot of production agent infrastructure actually behaves, and its weakness is distance: the policy lives in a different repository from the code that knows why the call failed, so it drifts every time a new error code is added, and the drift is silent because a missing entry falls back to "retry" or "don't" depending on someone's default.

That drift has a mechanism rather than a culprit, which is what makes it worth naming. A new error code is added by the person who found a new failure, inside the service, in the commit that handles it. The table is edited by whoever owns the harness, in another repository, on another day. Nothing links the two, so nothing fails when only the first happens, and the code ships classified only by a default. A default is a decision about failures nobody has looked at, applied to precisely the failures least understood, which is the worst place for a blanket answer. Putting the class in the response collapses the two edits into one, in the file where the failure is already being described, and a failure whose class was forgotten is then a failure whose body is visibly incomplete rather than one that silently inherits somebody's preference.

A third is a richer taxonomy — a dozen classes covering rate limits, quota exhaustion, authorisation, precondition failures, downstream degradation. Each additional class buys a distinct agent behaviour only if the agent can actually behave differently, and past three or four the extra classes collapse into the same three branches at generation time. The classes above are what a model reliably distinguishes; more granularity is useful to your dashboards and does not change a single tool call.

The test for whether a class earns its place is whether it selects a different action, not whether it names a different cause. Quota exhaustion and a rate limit have different causes and the same action — wait, then repeat the identical call — so as retry classes they are one class, and the difference between them belongs in the error code, which is where alerting and a dashboard read it anyway. Splitting them at the class level asks a caller to carry a distinction it cannot act on differently, and every such distinction is one more thing to get wrong at generation time in exchange for no change in behaviour. Three classes is not a claim that failures come in three kinds. It is a claim that a caller has three moves.

None of this guarantees compliance. The flag is a strong signal and not an instruction the model is obliged to follow, and a model that has been told twice in the system prompt to be persistent will occasionally retry a terminal anyway. Two things reduce that to a rounding error. The tool description should state the contract — "errors with class terminal will not succeed on repetition; use alternatives or report to the user" — so the semantics are in the model's context before the first failure rather than only after it. And the harness should enforce the same rule independently, refusing a repeat of an identical call that returned terminal and substituting a synthetic result that says so; belt and braces, because the cost of a hammering loop against a rate-limited third party is measured in suspended API keys rather than tokens.

A retry class is a small field that removes the one inference a model cannot make from prose. transient says the call is fine and the moment is wrong, and carries the number of seconds to wait. invalid_request says the arguments are wrong and names what would be accepted. terminal says this call is over, and carries whatever the domain can offer instead — a neighbouring slot, a tool that unblocks the condition, or nothing at all, which is still enough to stop four attempts and a false outage report.

Everything in this chapter assumed a failure path someone designed. A great many tools do not have one: they wrap a client library, let whatever it raises propagate, and hand the model whatever the framework serialises — which, more often than anyone would like to admit, is a stack trace.