Rounds grow input tokens quadratically while calls grow linearly

The contract tool's design document carried one line about cost: debate ensemble — roughly 3x single pass, acceptable at current volume. The reasoning behind the 3 was that a debate involves several calls where a single pass involves one, and three felt like the right order of magnitude for "several". The service processes about 3,000 agreements a month, the single-pass version had been billing around $265 a month, and $795 was signed off without much discussion. The first full month of the debate version came to $2,410.

Nothing went wrong. No runaway loop, no retry storm, no misconfigured model. The estimate was wrong twice, in the same direction, and the two errors compound.

Counting the calls is the correction that is still wrong#

A three-round debate between two agents produces six debater calls, and the judge adds a seventh. That is the first correction, and it takes the estimate from 3x to 7x on call count alone. It is the correction most people make when challenged, and stopping there leaves the larger factor untouched, because calls are not what providers bill.

What gets billed is tokens, and the token profile of a debate is shaped by two things that behave differently. Every call re-sends the shared material — the source document, the policy or instructions, the role prompt — which scales linearly with the number of calls. Every call after the first also carries the exchange so far, and that part scales with the square of the rounds, because turn k of the debate is sent again as context in turns k+1 through the judge's.

Writing out the sum makes the shape explicit. With two debaters, R rounds taken in sequence, a shared prefix of B tokens and turns averaging t tokens each:

def debate_input_tokens(B: int, t: int, R: int, judge_prefix: int) -> dict:
    """Total INPUT tokens for 2 debaters x R rounds + 1 judge."""
    turns = 2 * R
    # every debater call re-sends the shared prefix
    prefix = turns * B
    # turn k (0-indexed) sees k prior turns: sum(0..turns-1) = R*(2R-1)*t
    transcript = R * (2 * R - 1) * t
    judge = judge_prefix + turns * t
    return {
        "prefix": prefix,
        "transcript": transcript,
        "judge": judge,
        "total": prefix + transcript + judge,
    }

# contract tool: 40-page agreement + instructions, detailed cited turns
debate_input_tokens(B=27_400, t=1_200, R=3, judge_prefix=27_400)
# {'prefix': 164400, 'transcript': 18000, 'judge': 34600, 'total': 217000}

Against a single pass at 27,400 input and 400 output tokens, that run costs 217,000 input and 7,550 output tokens. At $3 per million input and $15 per million output, the single pass is 8.8 cents and the debate is 76.4 cents, which is 8.7x. Multiply by 3,000 agreements and you get $2,292, and the remaining $118 of the actual bill is retries: about 4% of calls came back failing JSON schema validation and were re-sent, and a failed judge call means re-sending the whole transcript.

The judge's call is worth singling out because it is the largest single call in the run — 34,600 input tokens against 27,400 for a single pass and 28,600 for the first debater turn. That is true only when the judge is given the source document. Chapter 3's pipeline omitted the document from the judge's prompt, which drops the judge's call to about 8,300 tokens and saves roughly 4% of the run, and buys that saving by making the judge structurally unable to check a quotation. It is a cost optimisation that removes the one thing that could have made the judge's verdict mean something.

The multiplier depends on how much of the prompt is shared#

The 8.7x above is specific to a long source document, where the shared prefix dominates and the quadratic term is only 8% of the input. Move the same protocol to a short input and the arithmetic inverts, because the exchange is now large relative to what is being argued about.

The support ticket column is the one that surprises people, and it is the one that matters for the classification work in the next chapter. When the thing being classified is 400 tokens long, six agents writing 700-token arguments about it produce an exchange seventeen times the size of the input, and the run costs nearly thirty times a single pass despite being seven calls. Cheap input does not make debate cheap; it makes the debate the expensive part.

Round count is where the quadratic term shows itself directly. Holding the ticket regime fixed and varying R:

Going from three rounds to six doubles the call count and multiplies the transcript re-send by 4.4, which is the arithmetic behind the observation in chapter 4 that the six-round experiments cost more than twice the three-round ones. Anyone proposing "let them argue until they converge" is proposing an open-ended quadratic.

Prompt caching changes these numbers more than any protocol tweak, and it is the first thing to reach for once you have decided the debate should exist. Caching here means the provider keeps the processed form of a prefix it has already read and re-uses it on the next call whose input starts with exactly the same text, charging a write the first time and a much cheaper read afterwards. The shared prefix is identical across calls that carry the same system prompt, so a provider cache bills it at roughly a tenth of the input rate on a hit, against about 1.25x on the write. In the contract regime with three distinct system prompts — pro, con, judge — you get three cache writes and four reads, and the prefix cost falls from 191,800 to 113,710 tokens, taking the run from 8.7x to 6.0x. Restructure so that all seven calls share one system prompt and the role is supplied in the user turn, and you get one write and six reads: 50,690 tokens of prefix, and the run lands at 3.9x. The original 3x estimate is nearly reachable, by a route nobody in that design review was thinking about.

Concurrency moves the latency and leaves the bill#

The usual response to the corrected figure is that the debaters within a round run in parallel, so the cost is fine. Two things are wrong with it, one arithmetic and one about protocol design.

Parallel execution does not reduce token count. Seven calls issued concurrently bill exactly what seven calls issued in sequence bill, so concurrency is an answer to a latency objection offered against a cost objection. Whatever the wall-clock improvement is, the invoice is unchanged.

The protocol point is the more interesting one. In the loop from chapter 1 the con agent's round-one turn is a rebuttal, which means it is written after reading pro's round-one turn. Those two calls cannot overlap, because the second one's input depends on the first one's output. Running them concurrently requires switching to a simultaneous protocol where both agents write round n from the state at the end of round n−1 and neither sees the other within a round. That is a change to what each agent is responding to, not a scheduling optimisation, and it weakens the mechanism the design was buying: an agent answering last round's argument is doing something looser than an agent answering the argument in front of it.

There is a second, quieter latency effect in the simultaneous version. A round finishes when the slower of its two calls finishes, so the round's latency is the maximum of two draws from the call-latency distribution rather than a single draw. With individual calls at a median of 3.2 seconds and a 95th percentile of 7.1 seconds, the maximum of two lands at a median near 4.1 seconds and a 95th percentile near 8.0, and three rounds compound that. A debate is therefore more exposed to provider tail latency — the slow end of the distribution of call times, which the 95th percentile summarises — than its call count suggests, because it takes the worst of two draws three times over and then adds a judge on top. Retry policy interacts with this badly: a single retried turn adds a full call to the critical path, the chain of calls that has to finish one after another before the run can return anything, and on a run that already sits at 17 seconds, one retry pushes it past most interactive budgets.

The sequential arrangement takes about 23 seconds per case against 3.4 seconds for a single pass. The simultaneous arrangement takes about 17 seconds, and the round duration is the slower of two calls rather than one, so a round's median is above a single call's median even though the two overlap. Concurrency bought six seconds and cost the rebuttal structure, which is a trade worth making deliberately rather than by default because someone objected to the price.

What a week of traffic tells you that a spreadsheet cannot#

Everything above is a model, and models of token spend are systematically optimistic because they assume mean-sized documents, no retries and no abandoned runs. The measurement that settles it is a week of production traffic with per-call accounting, which is a small amount of instrumentation.

def instrumented(call, *, run_id, role, round_no, **kwargs):
    t0 = time.monotonic()
    resp = call(**kwargs)
    emit("llm_call", {
        "run_id": run_id,
        "role": role,              # pro | con | judge | single_pass
        "round": round_no,
        "input_tokens": resp.usage.input_tokens,
        "cache_read_tokens": resp.usage.cache_read_input_tokens,
        "cache_write_tokens": resp.usage.cache_creation_input_tokens,
        "output_tokens": resp.usage.output_tokens,
        "latency_ms": int((time.monotonic() - t0) * 1000),
        "attempt": kwargs.get("attempt", 1),
        "outcome": "ok",
    })
    return resp

Group those rows by run_id and you get a per-case multiplier, and the number to report is a distribution rather than a mean. Document length in a real corpus is heavily skewed, and the contract tool's week came back at 6.1x median, 8.7x mean and 11.4x at the 95th percentile, with the tail driven by a handful of 90-page agreements where the prefix term dominates everything. A mean of 8.7x is the number for the finance conversation; the p95 of 11.4x is the number for the rate-limit conversation, because that is where the token-per-minute ceiling gets hit.

Four things belong in that week's report beyond the multiplier itself. The retry rate, counted as extra calls per run rather than as a percentage, because a retry on the judge is a different size from a retry on a first turn. The cache hit rate, since a prefix cache with a five-minute time-to-live, meaning the entry is discarded five minutes after it was last touched, misses on low-traffic services and the modelled saving evaporates. The abandoned-run rate, meaning cases where the pipeline timed out after four or five calls and produced nothing, which bills in full and returns no verdict. And end-to-end latency at p95 against whatever the service's latency budget actually is, because that is the constraint that usually bites before spend does.

What the cost argument does not establish#

None of this says debate is unaffordable. At 8.7x on a contract worth six figures, $2,410 a month is not the interesting number in the decision, and there are tasks where a 30x multiplier on a cheap classification is still trivial in absolute terms. The point of getting the arithmetic right is that it lets the value question be asked honestly, and a 3x estimate that is really 8.7x has already answered it by making the price invisible.

Token spend is also not the whole cost. Seven calls per case means seven opportunities for a provider error, seven contributions to a rate-limit budget shared with everything else in the account, and a failure mode where a run dies halfway and leaves a partial transcript that some part of the system has to decide what to do with. The operational surface grows with the call count even where the bill does not.

And the multiplier says nothing at all about quality. A design can be 8.7x and worth it, or 8.7x and worth nothing, and the arithmetic in this chapter cannot distinguish those cases. What it does is set the bar the quality measurement has to clear, which turns out to be the harder half, because the improvements debate actually delivers on real tasks are usually small enough that a 400-case evaluation cannot see them at all.