Four budgets, because each catches a runaway the others miss

The email-drafting agent has one job: read an inbound customer message, look up whatever the reply needs — order, subscription, shipment, past tickets — and write a draft for an agent to approve. It runs about 20,000 times a day, averages four tool calls per run, and finishes in 22 seconds at p50. On the Tuesday in question, someone deployed a change to the client for the CRM — the customer relationship management system, which is where this company keeps orders, subscriptions and past tickets — that dropped the read timeout, and the CRM began accepting connections and answering after six to seven minutes instead of 300 milliseconds. The runs that hit it made three CRM calls each and took just over twenty minutes apiece. They were not errors. Every call eventually returned a 200 with correct data, the agent produced a good draft, and the run terminated cleanly.

The step budget was set to 12 and those runs used 5. The token budget was set to 60,000 and those runs used about 31,000, which was normal. The money budget was set to $0.25 per run and those runs cost $0.04, which was also normal. Three limits, all configured, all watching, none of which had anything to say about a run that took fifty-five times longer than it should. What noticed was the queue: the worker pool sized for 22-second runs backed up by 4,100 messages in ninety minutes, the oldest unanswered customer email aged past the four-hour support SLA, the service level agreement that promises a first reply within a stated time, and the first signal anyone acted on was a support lead asking why the drafts had stopped arriving.

The four limits count four different things#

The reason a single budget cannot substitute for the others is that the resources an agent consumes are not proportional to one another. A step is a decision, a token is a unit of context, a second is elapsed reality, and a dollar is a price that changes underneath both of the first two. Each runaway mode saturates one of those while leaving the rest looking healthy, which is why omitting a limit does not leave you 25% less protected — it leaves one route to unbounded consumption completely open.

Three of the words in that sentence are doing technical work and are worth pinning down before the table that uses them. A budget here is a ceiling on something a run consumes, held by the harness and enforced by ending the run when it is crossed; it is not a warning and not a target. A step is one model round trip — the harness sends the transcript, the model replies, and whatever tool calls come back are dispatched — so steps and decisions are the same count. And the figures quoted for this agent are percentiles of its own measured history, where p50 is the value half its runs come in under and p95 the value that ninety-five runs in a hundred come in under. A percentile is a statement about this agent's traffic and about nothing else.

Take the three limits that were configured on the Tuesday and work out, one at a time, why each of them stayed quiet. The step budget counts model round trips, and a run that makes three CRM calls makes the same number of round trips whether each call returns in 300 milliseconds or in seven minutes; five steps is five steps. The token budget counts the context flowing in and out, and a slow response is not a large one — the CRM returned the same rows it always returned, so those 31,000 tokens were the run's ordinary 31,000 tokens. The money budget prices those tokens, which means it inherits the token budget's blindness exactly and contributes none of its own. Line the three expressions up and the informative thing is the term they have in common, which is the term none of them has: steps count decisions, tokens count context, dollars price the two of them, and not one of the three carries anything denominated in seconds.

That is what the incident licenses, and it fits in a line. A step, token or money budget certifies what a run consumed and never how long it took to consume it. All three were constant, by construction, while the one property that had changed varied by a factor of fifty-five — and a quantity that cannot move when the dangerous property moves is not a weak signal, it is not a signal at all. The converse is why the fourth limit is not optional: elapsed time is the only one of the four that a hang shifts, because a hang is precisely a failure defined by shifting nothing else.

The bottom two rows carry the parts of the argument that get left out most often. Wall clock is the only limit with a relationship to the humans waiting, and it is the one the Tuesday incident needed. Money is the only limit denominated in the thing the finance team actually asked about, and it is the only one that survives a change in what a token costs: the same run at the same 31,000 tokens costs $0.04 on the model this agent uses and $0.29 on the frontier model somebody might route it to during an experiment, and no token budget anywhere would register the difference. Priced tools make the gap wider still, because a search API at four cents a call puts $0.16 into a run that the token accounting values at zero.

Both halves of that are worth doing rather than reading, because they are the same absence approached from two sides. The priced-tool figure comes from multiplying the per-call price by the call count: this agent averages four tool calls per run, so a search API charging four cents a call adds sixteen cents to a run. The token budget's view of those sixteen cents is not a low estimate. It is zero, because a priced tool call is a payment that never appears in any count of input or output tokens, and a term that is absent cannot be inaccurate. The two model prices work the same way from the other direction: the token count is held fixed at 31,000 across both figures, so the entire difference between them is price, and price is exactly the term the token accounting does not carry.

The top row deserves a concrete case too, because "steps" is the limit people set first and then assume is redundant once the others exist. The runaway it owns is the cheap one: this agent has a list_recent_tickets tool that returns titles only and a get_ticket tool that returns one ticket, and a model that cannot find what it is looking for will alternate between them, twelve tickets deep, at roughly 700 tokens and 400 milliseconds per pair. Reaching the token budget that way takes about 85 steps, reaching the wall-clock budget takes about 110, and reaching the money budget takes longer than either. The step budget ends it at 12. Steps are also the only limit that bounds side effects rather than resources, which is the argument for keeping them tight on any agent whose tools write: a run permitted 50 steps is a run permitted to send up to 50 emails, whatever it costs.

The shape of that argument matters more than any of its arithmetic, and the shape is that a cheap oscillation is cheap in every currency except decisions. Each pair of calls returns a little text quickly and costs a little money, so every limit denominated in a resource has to wait for small amounts to pile up, and the piling up is slow precisely because the amounts are small. The step budget never waits, because a step is not a resource that accumulates towards a ceiling — it is the thing being spent, one per decision, and an oscillation spends one per decision from the very first pair onward.

Numbers you can defend#

Every number in that table came from the same procedure, which is worth stating because "cap it at 20 steps and 50,000 tokens" is a guess and a guess is what fails an incident review. Measure the distribution over a week of real runs, take p95, and set the limit at roughly 1.5 to 2 times it. For this agent that means 8 steps at p95 and a budget of 12, 26,000 tokens at p95 and a budget of 60,000, 48 seconds at p95 and a budget of 90. The multiplier is a judgement about what a legitimately hard case looks like: an email that needs seven lookups is unusual and real, an email that needs forty is not an email.

Check the three against that rule and one of them sits outside it, which is worth naming rather than smoothing over. Steps land at a multiple of one and a half above p95 and seconds a little under twice, both inside the stated band. Tokens land well above twice p95, and deliberately so, because the runaway the token budget owns is not a run at the top of the distribution — it is a single oversized tool result, which is a first-step event rather than a p95 event. A limit derived from the shape of a distribution protects against runs that are unusually long. A limit guarding against one enormous payload has to sit above anything the distribution contains, because the payload is not in the distribution at all. So the multiplier rule holds wherever the runaway is a long run, and the token budget is the case where that premise does not apply.

The budget that fires most often should fire on the order of 0.1% to 0.5% of runs. Above that, the limit is too tight and is truncating real work; below about 0.05%, the limit is loose enough that a moderate regression will hide underneath it, which is how you end up with a step cap of 50 that has never once fired and provides no protection at all against a run that needs 6. On this agent the step budget fires on about 0.2% of runs, almost all of them genuinely long threads, and every firing is a logged event with a reason code rather than a silent truncation.

Fleet-level money is a separate control from per-run money and needs its own number. Twenty thousand runs at $0.044 expected comes to roughly $880 a day, so the daily cap sits at $1,400 with an hourly cap of $110 underneath it, because a burst that will blow the day is visible in an hour and waiting for midnight to find out is not a control. The two caps also fire differently. Killing a run mid-flight because the fleet's daily spend crossed a line punishes an arbitrary customer for something their email did not cause, so the fleet cap stops admission at the queue and lets in-flight runs finish; the per-run cap is the one allowed to interrupt work in progress.

Dividing one of those caps by the other is the thing to head off, because a day has twenty-four hours and a pro-rata slice of $1,400 would be a far smaller hourly figure than $110. The hourly cap is not a share of the daily one and is not meant to be. Expected spend is about $880 across a whole day, so an hour that reaches $110 is an hour running well above the daily average rate, which is what makes it a detector: set high enough that ordinary traffic never touches it, and low enough that a burst large enough to threaten the day announces itself inside the hour rather than at midnight. Naming it a detector also settles what it should do when it fires, which is stop admission and raise an alarm, and not any arithmetic about subtracting the overspend from the hours that remain.

Where the checks have to live#

A budget that is only checked at the top of the loop cannot see anything that happens inside a step, which is precisely the Tuesday failure: the run was blocked inside a socket read for six minutes and the loop condition was not being evaluated at all. Wall clock in particular needs the deadline pushed down into the calls it bounds.

@dataclass
class Budget:
    max_steps: int = 12
    max_tokens: int = 60_000
    max_seconds: float = 90.0
    max_cost_usd: float = 0.25

    steps: int = 0
    tokens: int = 0
    cost_usd: float = 0.0
    started: float = field(default_factory=time.monotonic)

    def elapsed(self) -> float:
        return time.monotonic() - self.started

    def remaining_seconds(self) -> float:
        return self.max_seconds - self.elapsed()

    def check(self) -> str | None:
        """Returns the name of the exhausted budget, or None."""
        if self.steps >= self.max_steps:
            return "steps"
        if self.tokens >= self.max_tokens:
            return "tokens"
        if self.elapsed() >= self.max_seconds:
            return "time"
        if self.cost_usd >= self.max_cost_usd:
            return "money"
        return None

    def tool_timeout(self, tool_default: float = 15.0) -> float:
        # No single call may outlive the run it belongs to.
        return max(0.5, min(tool_default, self.remaining_seconds()))

    def would_exceed_tokens(self, incoming: int) -> bool:
        return self.tokens + incoming > self.max_tokens

tool_timeout is the line that would have caught the CRM hang. A call made at second 40 of a 90-second run gets 15 seconds because that is its own default; a call made at second 84 gets 6, because the run has no business continuing past its deadline in order to wait for a tool it will not have time to use. The alternative — a watchdog thread that cancels the run from outside — is a fine belt to go with these braces, and it is the only thing that helps when the blocking call is in a C extension that ignores signals, but propagating the deadline is what makes the common case correct.

That expression is short enough to read as algebra, and each of its two operators answers a different question. The inner min compares the tool's own default against whatever is left of the run and takes the smaller, which enforces a rule with no exceptions in it: no single call may outlive the run it belongs to, because a call returning after the run has stopped returns into nothing, and every second it spent was spent for no possible benefit. The outer max puts a floor under the result. Without that floor, a call issued a moment before the deadline would be handed a timeout of zero or less and would fail on the spot, which converts the final instant of every run into a guaranteed failure rather than a slim chance of success. Neither operator is a tuning knob. Each is the only sensible answer to its question, which is why the same two lines turn up in every client that propagates a deadline.

would_exceed_tokens is checked before a tool result is appended rather than after. This matters because the 4 MB CSV becomes a token cost the moment it enters the message list, and once it is there every subsequent step pays for it again; catching it pre-flight means the result can be truncated to the first 200 rows with a note saying so, and the run continues. Checking after the fact means the budget reports an overrun you have already paid for twice.

The arithmetic under that is the same arithmetic as the compact function in the first chapter of this topic, and it is worth doing once properly. A tool result that enters the message list is resent with every later request, so its true cost is its own size multiplied by the number of steps that follow it, and not its size. Admitting a large result early therefore costs a great deal more than admitting the same result late, and the run has no mechanism for giving those tokens back afterwards. Checking before the append is what keeps truncation available, because truncating is only an option while there is still a choice about what gets stored. Checking afterwards leaves reporting as the only thing the check can do, and by then the number being reported has been paid at least twice — once on the way in, and once on the resend.

budget.steps += 1
if (exhausted := budget.check()):
    return terminal(reason=f"budget_exhausted:{exhausted}", budget=budget,
                    messages=messages)

response = client.messages.create(..., messages=messages,
                                  tools=breaker.available(TOOLS))
budget.tokens += response.usage.input_tokens + response.usage.output_tokens
budget.cost_usd += price(response.usage, MODEL)

Cost is accumulated from the provider's reported usage rather than estimated from a token count, because the two diverge exactly where it matters most. Prompt caching means a cached read can be a fraction of the price of the same tokens uncached, and a cache-hit rate that quietly drops from 85% to 0% after a system-prompt edit produces a run with identical step and token counts costing several times more. That regression is invisible to three of the four budgets and obvious to the fourth.

That case is the cleanest argument for the fourth budget, and it has the same form as the Tuesday hang with a different term missing. A cache-hit collapse changes nothing a step or token counter can observe: the same requests are made, carrying the same tokens, in the same order, and both counters are functions of exactly those things. What moves is the multiplier the provider applies to tokens it has already counted, and a budget reading the token count has no term for a multiplier applied to it. So the token budget will report a healthy run at any price whatever, which is not a matter of the threshold being set too high — no threshold on a count can bound a product whose other factor the count does not contain.

Where a budget's protection ends#

They are not a rate limiter. A rate limiter protects a dependency from your aggregate traffic and lives at the fleet boundary; a budget protects the run's own resource consumption and lives in the loop. The two get confused because both refuse work, and they refuse different work for different reasons — a run that is well inside every budget can still be one of four hundred hammering a search API that permits fifty.

They are not a quality control either. A run that finishes in 5 steps, 22,000 tokens and 19 seconds is a run that stayed inside its allowance, and it can still have produced a draft that tells the customer their refund was processed when it was not. Nothing in this chapter looks at output, and the last chapter of this topic is about how easily budget-green becomes a proxy for correct.

The procedure for arriving at the numbers has an edge it does not cover, which is an agent that has no history to take a percentile from. A limit derived from a distribution needs a distribution to exist, and on the first day there is not one. The workable answer is to start with limits set generously and firing loudly, treat each firing as a case to be read rather than a run to be truncated, and tighten once a week of real traffic has accumulated — which means accepting that the first week's protection is against catastrophe and not against regression. What does not work is copying the numbers from a different agent, because every figure in the table above is a property of this agent's tool set and this agent's task. The multiplier is the only part of the procedure that travels.

What the four limits amount to, taken together, is four questions asked of one run, and the reason to ask all four is that each is deaf in a direction the others hear. Steps hear decisions and are deaf to size. Tokens hear size and are deaf to time. Wall clock hears time and is deaf to price. Money hears price, and being downstream of the other three it is the last to learn anything they have already missed. Dropping one of them does not thin the protection evenly across every failure mode; it opens the single route that the dropped limit was alone in watching, which is how three configured budgets watched a run take twenty minutes and had nothing to report.

The honest limitation is that a budget is a backstop and backstops charge full price. When the step cap fires at 12, the run has already made 12 model calls and paid for all of them; when the money cap fires at $0.25, that quarter is spent. If a run has been alternating between the same two tool calls since step 4, the budget will end it at step 12 having paid eight steps of rent on a state that was already detectably stuck. Recognising that state directly, at the moment it repeats rather than at the moment the allowance runs out, is a cheaper instrument and a different one.