Reasoning allowance regimes

A reasoning token is a token a model generates for itself before it writes the reply — deliberation, billed at the output rate, usually never rendered to anyone — and a thinking budget is the request parameter that caps how many of them a single call may produce. Both of those definitions hold across providers. What does not hold is the accounting behind them: which allowance the deliberation is charged against, and therefore whether max_tokens, the limit on the completion, has any bearing on it at all. A budget of 2,000 against a max_tokens of 1,024 is a contradiction one API rejects at validation, a legal request on the next that bills up to 3,024 tokens of output, and an unrecognised parameter on the third.

Two accountings, and a third arrangement with no budget at all#

Where max_tokens covers thinking plus visible output, the two settings compete for a single pool, so authorising more deliberation than the pool contains is something a validator can refuse from the integers alone. Where reasoning is metered separately, the reasoning cap and the visible-output cap are limits on two different things that happen to appear in the same request body. Nothing about budget 2000, max_tokens 1024 is contradictory there: it means up to 2,000 tokens of deliberation, then up to 1,024 tokens of answer, for a possible 3,024 tokens of billed output from a request whose only visible limit says 1,024.

That worst case is worth arriving at rather than accepting, because it is the number a migration gets wrong. Two thousand tokens of deliberation is what the reasoning cap permits, one thousand and twenty-four tokens of answer is what max_tokens permits, and on a separate allowance nothing connects the two counters, so the arithmetic that bounds the call is addition: two thousand plus one thousand and twenty-four, which is the 3,024. On a shared allowance the same two integers cannot be added, because there are not two spans to add — the deliberation and the answer are stretches of one sequence and one counter runs over the whole of it, so the most a call can bill is max_tokens and no configuration raises that. Three times the billed output for the same request body, decided entirely by which side of that distinction the endpoint sits on.

The distinction also decides what a validator is able to do for you, and this is where the two regimes part company in a way that shows up in an incident report rather than in a schema. A budget above max_tokens on a shared allowance is a contradiction contained inside the request: the two integers disagree with each other, so a comparison of two fields on the way in settles it, and the caller gets a 400 before a model is reached. On a separate allowance those same two integers do not disagree about anything. They describe two independent limits, both satisfiable, both satisfied. There is no contradiction to detect, so no validator anywhere in the stack can raise one, and the request that overspends is not malformed — it is well-formed, accepted, answered correctly, and expensive. That is the whole mechanism behind the "silent overspend" row of the table below: silence is not an oversight in the API, it is what a correct API does with a legal request.

There is also a third arrangement, increasingly the default on newer model families, where the integer budget is gone. Anthropic's Opus 4.6 and later removed budget_tokens outright — sending it now returns a 400 — and replaced it with adaptive thinking, where the model chooses its own depth of deliberation per request, plus an effort level from low through max; OpenAI's reasoning models take reasoning.effort rather than a token count. In that arrangement you bias the model's decision rather than bounding it, which means there is no hard ceiling on reasoning at all. What you still have is the pool: max_tokens continues to bound thinking and visible output together, so the subtraction from chapter 2 — where the answer is guaranteed only the pool minus whatever deliberation takes — still applies, with a term you can no longer pin down.

An effort level is not a soft budget, and reading it as one is the mistake this regime invites. A budget was enforced by a counter: the deliberation was measured as it was produced and compared against an integer, and when the comparison succeeded the model was pushed out of thinking and into answering. Remove the integer and there is no comparison, so there is nothing to enforce — low does not mean at most this many tokens, it means the model is disposed to deliberate less. The one hard limit still standing is max_tokens over the pair, and hitting that limit does not curtail the deliberation, because the deliberation came first in the sequence and had already been generated and billed. It curtails the answer. So the regime that removed the reasoning ceiling did not move the enforcement elsewhere; it left the request with a control that steers and a control that truncates, and neither of them stops the model spending.

The divergence is not arbitrary, and knowing where it comes from makes it easier to predict which regime a new API will be in. A shared pool falls out of treating the completion as one sequence: the model generates deliberation and answer through the same decoder into the same buffer, that buffer has to fit alongside the prompt inside the context window, which is the fixed span of tokens the model can hold in one request, and a single limit on the whole thing is the smallest honest description of the constraint. Separate metering falls out of treating thinking as a distinct phase with its own configuration surface, which reads more naturally in a request body and hides that the two still share a context window. The first is truthful about the mechanism and awkward to configure; the second is comfortable to configure and quietly misleading about what your limits protect.

The move to effort levels is a reaction to a weakness both share. A fixed integer budget is the wrong instrument for a workload with a wide difficulty spread, because a value large enough for the hard tenth of requests is wasted on the easy nine tenths, and a value tuned for the median starves the tail — which is the correlation between input difficulty and reasoning length from chapter 2, restated as a configuration problem. Adaptive thinking moves the decision to the model, per request, where the information about difficulty actually is. What you give up is the ceiling, and with it the ability to answer "what is the most this call can cost" from configuration alone, which is a real loss for anyone whose spend model was built on that guarantee.

The rows that matter operationally are the fourth and fifth. On a shared allowance, max_tokens is a genuine cost ceiling for the call, and the thing you are protecting is the answer. On a separate allowance, max_tokens protects nothing on the cost side, and a team that has internalised "the output limit caps my spend" from one provider will carry that belief across a migration without noticing that it stopped being true.

That belief is worth defending before dismissing it, because it is not careless. On every API anybody integrated before reasoning existed, the output limit did cap the spend, for the plain reason that output was the only thing generated and the limit was on the whole of it. The belief is a correct generalisation from a decade of correct instances. What breaks it is not a change to max_tokens but the arrival of a second kind of generated token that some APIs decided to count somewhere else. The sentence to check on any new endpoint is therefore not "does this limit my output" — it does — but "is everything I am billed for counted as output here".

The reason a broken version of that belief runs for weeks rather than minutes is that the failure has no symptom in any of the three places a service is watched. Status codes stay in the success range, because a request that deliberates for thousands of tokens is a valid request that succeeded. Quality metrics stay clean, because the visible answer is correct — the deliberation improved it, if anything. Latency rises, but a latency alarm where one exists is a threshold on the endpoint's own history, and a threshold set against a fast endpoint's normal day is not necessarily crossed by a workload that is still returning inside its timeout. Cost is the one signal that moves decisively, and it is aggregated by day at the account level, where one workload's fourteen hours is diluted by everything else the account did. Every general-purpose detector is looking at a quantity this failure does not disturb, which is the whole argument for the per-call usage payload: it is the only place the spend appears attached to the call that caused it.

Three probes that settle it in ten minutes#

You can read this out of the documentation, and you should, but the API tells you directly and the API is what is actually deployed. Three probes settle it. The helper below automates the first of them and reads both of the reactions it can produce; the other two are a mis-sized request and an ordinary one, run by hand against the same model.

def classify_regime(call, *, model):
    """Send two deliberately mis-sized requests and read the reaction."""
    try:
        r = call(model=model, max_tokens=1024, reasoning_budget=2000,
                 prompt="Compare three sorting algorithms by cache behaviour.")
    except InvalidRequest as exc:
        return "shared allowance", str(exc)
    except UnknownParameter:
        return "effort only", "no integer budget accepted"

    visible = r.usage.output_tokens - r.usage.reasoning_tokens
    if r.usage.output_tokens > 1024:
        return "separate allowance", (
            f"billed {r.usage.output_tokens} output tokens "
            f"({r.usage.reasoning_tokens} reasoning) under max_tokens=1024"
        )
    return "shared allowance", f"clamped at 1024; visible text {visible} tokens"

The first probe is the contradictory configuration. A 400 before generation means the pool is shared. Acceptance means it is not, and the response then carries the proof: if output_tokens comes back at 2,400 on a request whose max_tokens was 1,024, the reasoning was drawn from somewhere max_tokens does not reach. A rejection naming the parameter as unknown rather than as too large puts you in the third regime.

Reading the status code alone is where that probe goes wrong, and the error message is what saves it. A shared-pool provider commonly enforces a floor on the budget as well as the relation to max_tokens — chapter 2's rejection of a budget beneath the provider's minimum is the same 400 with a different sentence in it — and a request can also be refused for a model name, a region, or a parameter the endpoint has since retired. Three different faults, one status code. What distinguishes them is that only one of them names both integers in the same breath. Read the message, and record it, because the message is the evidence that the pool is shared rather than the code.

The subtraction two lines below the try block carries an assumption worth making explicit, because it is the kind that produces a plausible number rather than an error. Computing the visible length as output_tokens minus reasoning_tokens presumes that the reasoning is already counted inside output_tokens — which is the reporting convention a shared allowance implies, since there is only one span of generation to count. On a separate allowance that presumption need not hold, and where reasoning is reported alongside the output count rather than within it, the same subtraction returns a number that is too small and occasionally negative. That is exactly why the value is read only on the branch that has already concluded the pool is shared. A quantity that is meaningful in one regime and misleading in another has to be computed after the regime is known, not before.

The second probe is the usage payload on an ordinary request, the same per-call token accounting that chapter 1 established for cost reconciliation, read here for a different purpose. Sum the reported reasoning and the tokens in the visible text; if that sum can exceed the configured limit, the allowances are separate, and if it is always clamped exactly at the limit, they are shared.

The word doing the work in that sentence is can, and it is the reason one request settles nothing. A shared pool has a single counter, so the sum it produces is bounded by the limit on every request without exception, and a sample of easy requests that all come in well under the limit is consistent with both regimes. What separates them is the tail. Send inputs hard enough that the model deliberates at length, and watch what the totals do as they approach the limit: under one counter they stop dead at it, in the exact sense that the sum equals the limit and the finish reason says why, and under two they walk straight past it. Landing exactly on a limit is not a coincidence that happens twice, which is what makes the clamp a signature rather than an observation.

The third probe is what happens when you squeeze. Set the limit deliberately low — 256 tokens with reasoning on — and look at the shape of the failure. A shared allowance returns empty or near-empty content with a length-based finish reason, because deliberation consumed the whole budget before the answer began. A separate allowance returns a complete-looking chunk of reasoning accounting and a visible answer chopped at 256 tokens, which is an ordinary truncation and tells you nothing about reasoning at all.

That third probe needs a control, because an ordinary truncation is what a tight limit produces on any model whether it deliberates or not. Run the same squeezed request twice, once with reasoning enabled and once with it disabled, and compare the visible text. On a shared allowance the two differ sharply: the version without reasoning spends all 256 tokens on prose and returns a paragraph, while the version with it returns little or nothing, because the prose was competing with deliberation for the same 256 and lost. On a separate allowance the two look alike, since the visible limit was never the thing reasoning drew from. The comparison is what turns "the answer was short" into evidence about where the tokens came from.

Configuration that cannot be misread#

Knowing the regime is worth little if the knowledge lives in a wiki page rather than in the code, and the shape that puts it in the code is an intent struct the caller fills in and an adapter that does the arithmetic. What the caller states is what it wants: the worst-case answer it is willing to serve, and the ceiling it will tolerate on hidden deliberation. Neither of those is a statement about any particular vendor's parameter names.

@dataclass(frozen=True)
class Budget:
    visible_tokens: int      # worst-case answer you are willing to serve
    reasoning_tokens: int    # ceiling on hidden deliberation

class SharedPoolAdapter:
    def params(self, b: Budget) -> dict:
        return {
            "max_tokens": b.visible_tokens + b.reasoning_tokens,
            "thinking": {"type": "enabled", "budget_tokens": b.reasoning_tokens},
        }

class SeparatePoolAdapter:
    def params(self, b: Budget) -> dict:
        return {
            "max_output_tokens": b.visible_tokens,
            "reasoning": {"max_tokens": b.reasoning_tokens},
        }

class EffortOnlyAdapter:
    LEVELS = ((2_000, "low"), (8_000, "medium"), (32_000, "high"))

    def params(self, b: Budget) -> dict:
        effort = next(
            (name for cap, name in self.LEVELS if b.reasoning_tokens <= cap), "max"
        )
        return {
            "max_tokens": b.visible_tokens + b.reasoning_tokens,
            "output_config": {"effort": effort},
        }

Budget(visible_tokens=2600, reasoning_tokens=6000) now means the same thing everywhere it is written, and each adapter is the single place where the regime is encoded. The effort adapter is the honest one about its own limits: it approximates an integer intent with a coarse level, so the reasoning ceiling it produces is a hope rather than a guarantee, and the comment in the sizing rule should say so.

Walking that one struct through all three adapters is what shows how much the regime changes and how little the caller has to know. On the shared adapter, max_tokens becomes the sum of the two fields and the budget becomes 6,000, so the answer's guarantee is the pool minus the budget — the sum, less the 6,000 — which comes back to the 2,600 the caller asked for. On the separate adapter the visible limit is set to 2,600 directly and the reasoning cap to 6,000, and the answer's guarantee is 2,600 because nothing is drawn from it; the worst-case bill is the same sum as before, reached by adding two spans rather than by filling one pool. Two regimes, two very different request bodies, identical guarantees, identical worst case. On the effort adapter, 6,000 is above the low cap of 2,000 and not above the medium cap, which is four times it, so medium is selected and max_tokens is again the sum. That is where the equivalence stops. Nothing in that request bounds deliberation at 6,000, so a medium-effort call that thinks past it eats into the 2,600 and the answer is truncated to whatever is left. The struct that guarantees the caller 2,600 tokens of answer on two providers guarantees nothing at all on the third, and the only visible difference between the three call sites is which adapter was constructed.

The coarseness is worth seeing in the same terms, because it is not a rounding error in an unimportant place. The level is chosen by the first cap in the list the ceiling does not exceed, so a ceiling of exactly the low cap selects low and a ceiling one token above it selects medium — and medium's cap is four times low's. A caller who nudges a reasoning ceiling upward by a token has not nudged anything; they have moved to a level whose whole range is four times as wide. That is the price of giving the decision to the model: your intent survives as a bias, and the granularity of a bias is set by whoever named the levels.

The sizing rules follow the table. On a shared allowance, size the pool for the worst-case answer stacked on top of the reasoning ceiling, and treat max_tokens as the per-call cost cap it genuinely is. On a separate allowance, the two limits move independently, so max_tokens becomes purely a quality-of-answer setting and the reasoning cap becomes the only thing standing between you and unbounded hidden spend — which means it must be set explicitly on every call. Several APIs in this regime default the reasoning cap to a generous value, or accept -1 for dynamic, and a default of that kind is how a classification endpoint ends up deliberating for thousands of tokens over a one-word answer. On an effort-only model, set the effort level from an evaluation sweep rather than from taste, keep max_tokens sized as a pool, and accept that your remaining defence is a spend alert rather than a limit.

Defaults, hosting layers and the context window#

The regime belongs to the model family and API surface, not to the vendor's logo. The same provider can ship one family that takes an integer budget from a shared pool and another that takes only an effort level, and a hosting layer in front of a model — a cloud marketplace, an aggregator that normalises everyone's parameters into one schema — can present a knob that is translated rather than passed through. Probe the exact model identifier and the exact endpoint you deploy against, and re-probe when either changes.

Knowing your regime also does not tell you what the defaults are, and the defaults are where the money goes when nobody passes a value. Write down, for each model you call, what happens when the reasoning parameter is omitted entirely: on some models reasoning is off and the omission is free, on others the omission means adaptive and the model spends what it likes, and on at least one family a dynamic setting is the documented default. Those three outcomes are indistinguishable in a code review of a call site that simply does not mention reasoning, which makes the omitted parameter harder to audit than a wrong one.

Nor does the regime say anything about how reasoning interacts with the context window across turns, which is a separate accounting with separate rules. Within the turn that generates it, deliberation occupies context like anything else the model writes, so a 6,000-token budget consumes 6,000 tokens of window before the answer starts. Between turns, most APIs drop it, so it does not accumulate the way visible history does. The exception is narrow and mandatory, and it is what the second half of this topic is about.

Three regimes, then, and one question worth asking before any limit is chosen: does the visible-output limit on this endpoint bound the deliberation or not. Where it does, max_tokens is a real cost ceiling and the number to protect is the room left for the answer. Where it does not, the reasoning cap is the only spend control in the request. Where there is no integer at all, the defence moves out of the request body and into an alert. The probes settle which of the three a given model and endpoint is in, and the adapter keeps that answer in one place.

What that knowledge buys is the ability to predict the failure before it arrives. On a shared allowance, an over-tight configuration surfaces as answers that stop early or never start, and it surfaces on exactly the requests that were working hardest. On a separate one, nothing surfaces at all until an invoice or a usage query is read, because every response is complete and well-formed and simply cost more than anyone expected. The first of those reaches an application as a handful of symptoms that look like several unrelated bugs from the outside, and telling them apart quickly is the difference between a fifteen-minute diagnosis and a two-day one.