Grouping calls under a correlation id

A correlation id is a value generated once, at the boundary where a human action enters the system, and stamped onto every record produced by the work that follows. Distributed tracing settled on the mechanism twenty years ago, Google's Dapper paper described it in 2010 as the only practical way to reassemble a request that fans out across services, and the W3C standardised the header as traceparent. Cost attribution is the same problem with money in place of latency: a priced row is a fact about one model call, and almost every question anyone asks about spend is a fact about a set of them. That set has no natural boundary in the data unless somebody puts one there.

What one click leaves behind#

A user clicks Research at 09:14:02.118. The HTTP response goes back at 09:14:31, and the work finishes at 09:15:11. In between, the priced rows land like this:

09:14:02.4  planner            claude-opus-5     1,904 in    612 out   $0.0248
09:14:04.9  subagent-1 call 1  claude-haiku-4-5  3,110 in    488 out   $0.0056
09:14:05.1  subagent-2 call 1  claude-haiku-4-5  3,088 in    501 out   $0.0056
09:14:05.1  subagent-3 call 1  claude-haiku-4-5  2,970 in    463 out   $0.0053
09:14:05.2  subagent-4 call 1  claude-haiku-4-5  3,201 in    455 out   $0.0055
09:14:05.2  subagent-5 call 1  claude-haiku-4-5  3,140 in    470 out   $0.0055
09:14:08.7  subagent-2 call 2  claude-haiku-4-5  6,402 in    901 out   $0.0109
09:14:09.0  subagent-4 call 2  claude-haiku-4-5  6,880 in  1,020 out   $0.0120
09:14:11.3  subagent-4 call 3  claude-haiku-4-5  9,455 in    788 out   $0.0134
09:14:14.8  (429, retried)     claude-haiku-4-5      0 in      0 out   $0.0000
09:14:17.2  subagent-1 call 2  claude-haiku-4-5  7,004 in    835 out   $0.0112
09:14:19.6  (429, retried)     claude-opus-5         0 in      0 out   $0.0000
09:14:23.1  synthesis          claude-opus-5    22,410 in  2,880 out   $0.1841
09:15:11.4  summariser (async) claude-opus-5    24,880 in  1,455 out   $0.1608

Fourteen rows, $0.454 total. The five sub-agents are separate model-driven workers the planner started, each given one facet of the question to research on its own and each making one to three model calls before reporting back. Every row is correct in the sense the previous chapter established — right model, right counts, right rate in force. What no query over those rows can produce is the number 0.454, because nothing on any row says that these fourteen and not some other fourteen belong to that click.

Two of the rows carry zero tokens; they are the 429s, the HTTP status a provider returns when you have exceeded its rate limit, which the client waits out and sends again. They cost nothing and are still worth seeing, because a feature that retries 14% of its calls has an availability story attached to its cost story. The last row, landing sixty-nine seconds after the click and accounting for 35% of the total, was written by a queue consumer in a different process that has never heard of the click at all.

That 14% is two rows in fourteen, and the trace can produce the figure at all only because the rejected attempts were written down. A wrapper that retries internally and writes one row per call that eventually succeeded would show twelve rows and no retries whatsoever, and the feature would look like one that never meets a rate limit. The rejected attempts cost nothing in tokens, which is exactly why the temptation to skip writing them is strong and exactly why skipping them is a mistake: their count is the earliest warning available that a feature is about to start failing rather than merely waiting.

The reason nothing on those rows says which fourteen belong to the click is worth following link by link, because the conclusion is stronger than "the data is inconvenient". Start with what the provider sends back at the end of a call. A response is a statement about one HTTP exchange: which model served it, how many tokens went in and came out, why generation stopped. The provider never saw the click, holds no representation of your features or your users, and has no field in which the answer could be reported even in principle. Nothing about the cause of a call travels back from the thing that served it.

So the association between the click and the call exists in exactly one place, which is the memory of the process that made the call. A handler up the stack knows which user pressed which button, a function down the stack is assembling an HTTP request, and the association is whatever variables happen to be in scope along the path between them. That is a genuine fact about the running system for the duration of the request and for no longer.

The duration is the whole problem. When the handler returns its response the frame unwinds, the locals go out of scope, and the association is gone — not archived, not degraded, gone, because nothing wrote it anywhere. What survives is the row on disk, and a row holds what its writer chose to put in it. A fact that was true in memory and never serialised cannot be read back out of the artefacts afterwards by any amount of cleverness, for the same reason a stripped binary cannot be asked what its local variables were called.

That is why the identifier has to be minted upstream and threaded down, and why nothing applied later will substitute for it. It has to exist before the fan-out, because after the fan-out the unit it names has already dissolved into calls that no longer resemble one another. It has to be minted at the point where the action enters the system, because that is the only place which both knows an action happened and knows that nothing has branched yet. The same chain accounts for what it costs to add attribution to a service that has been running without it: there is no column to backfill from, no log to reprocess and no join to write, because the information was never in the database in any form. The change is a code change in every path that can reach a model call, which is why it is routinely estimated as a reporting change and delivered as a project.

Why the timestamp is not enough#

The reflex fix is to group by user over a time window: take every row for user_id = u_8812 within five minutes of the click and call that the request. It works in a demo, where one user does one thing at a time and every call lands before the response.

The reflex is tempting because it is almost a valid inference. The calls really were caused by the click, and an effect really does follow its cause in time, so the rows really do cluster after it. The window reads that implication backwards. Following the click in time is something every row caused by the click does, and equally something every row caused by anything else in those seconds does, so nearness in time separates the two not at all. What a window measures is co-occurrence, and co-occurrence is symmetric where causation is not.

The window fails on the first user who has two requests in flight. If the same user clicks Research at 09:14:02 and Draft reply at 09:14:09, both features' rows fall inside both windows, and every row gets counted twice or assigned to whichever click the query happened to anchor on. Concurrency is not exotic — a user with two browser tabs open produces it, and so does any product with a background refresh. Measured on one production system, 22% of requests overlapped another request from the same user within a sixty-second window during business hours.

It also fails on the tail. The summariser above lands at 09:15:11, so a five-minute window happens to catch it; a sixty-second window would not, and no window catches a nightly enrichment job. Both failure modes push the same way. Overlap double-counts cheap synchronous calls into whichever request the window anchored on, and truncation drops the asynchronous tail — and the asynchronous tail is where the expensive model runs, because work that got deferred is usually work that was too slow to do inline. A number that is wrong in a random direction gets noticed when someone spot-checks it. A number that is wrong in one direction gets believed.

That direction is structural rather than an accident of this feature, and the reason is that a window has exactly one knob. Overlap and truncation are both governed by its width, and they move against each other. Widening it far enough to reach a summariser arriving sixty-nine seconds later also admits more of whatever else the user was doing in that minute. Narrowing it far enough to exclude the neighbouring click also amputates the tail. No width satisfies both demands, because there is only one width to set.

The two errors are also not the same kind of error, which is what settles the sign. Overlap misplaces money: a row belonging to one request gets counted under another, or under both, so any single request's figure can come out high or low depending on what else that user happened to be doing. Truncation misplaces nothing. A row falling outside every window is attributed to no request at all and leaves the sum entirely. Only one of the two can move a total, therefore, and the rows it removes are the deferred ones — so the shortfall accumulates across a day of traffic instead of averaging out of it.

Where the identifier has to reach#

The identifier is generated once, at the outermost boundary a human action crosses — the HTTP handler, the WebSocket message handler, the cron tick. It is not generated per model call, per sub-agent or per retry, because the whole point is that it is the same across all of them.

Getting it onto every row without threading a parameter through forty function signatures is what ambient context is for. In Python that is contextvars, which propagates across await boundaries and into tasks spawned from the current context:

from contextvars import ContextVar
from dataclasses import dataclass
import uuid

@dataclass(frozen=True)
class CallContext:
    correlation_id: str
    user_id: str
    tenant_id: str
    feature: str

_ctx: ContextVar[CallContext | None] = ContextVar("call_ctx", default=None)

def enter_request(user_id: str, tenant_id: str, feature: str) -> CallContext:
    ctx = CallContext(f"r_{uuid.uuid4().hex[:12]}", user_id, tenant_id, feature)
    _ctx.set(ctx)
    return ctx

async def call_model(client, **kwargs):
    ctx = _ctx.get()
    if ctx is None:
        raise RuntimeError("model call outside a request context")
    attempt = 0
    while True:
        attempt += 1
        try:
            resp = await client.messages.create(**kwargs)
        except Exception as exc:
            write_cost_row(ctx, kwargs["model"], usage=None,
                           attempt=attempt, error=type(exc).__name__)
            if attempt > 3:
                raise
            continue
        write_cost_row(ctx, resp.model, usage=resp.usage, attempt=attempt)
        return resp

Two details in that snippet do most of the work. The retry loop lives inside the instrumented wrapper, so a retried call writes two rows under one correlation id rather than one row that quietly represents two billable attempts. And a missing context raises rather than defaulting to unknown, because a default is how a system ends up six months later with 8% of spend attributed to a bucket nobody can decompose. Failing loudly in development is cheap; a growing unknown bucket in production is not.

It is worth being precise about why ambient context works at all, because the same property says exactly where it stops working. A ContextVar is a slot in the running process's memory, attached to the current task and copied into any task spawned from it. Every propagation in the diagram above that succeeds does so by staying inside that memory: an ordinary function call keeps the same frame stack, an await resumes the same task, and a sub-agent started as a child task inherits a copy of the context as it stood when the child was created. None of that is transmission. It is the absence of transmission — the value never had to move anywhere, because nobody left the process.

The queue boundary is where this breaks in practice, and it breaks because it is the one hop where the context genuinely does not travel by itself. A contextvar does not cross a Redis queue, a Postgres job table or an SQS message. The correlation id has to be written into the job payload at enqueue time and re-entered at the top of the consumer:

def enqueue_summarise(doc_id: str) -> None:
    ctx = _ctx.get()
    queue.publish({"doc_id": doc_id, "_ctx": asdict(ctx)})

def consume(msg: dict) -> None:
    _ctx.set(CallContext(**msg["_ctx"]))   # re-enter, do not re-generate
    summarise(msg["doc_id"])

If the consumer generates a fresh correlation id instead of re-entering the enqueued one, the async row is technically attributed — to a phantom request with one call and no user click behind it — which is worse than being unattributed, because it looks fine on a dashboard. Auditing this is a two-line check: count rows whose correlation id appears exactly once on a feature you know fans out, fan-out being the shape the trace above has, where one user action spawns many model calls across several workers rather than making a single call and returning. On such a feature that count should be zero, and any drift above zero is a propagation hole.

Four fields, four questions#

Four identifier fields do four different jobs, and collapsing any two of them loses a question you will eventually be asked.

What separates them is grain — the number of rows that are expected to share a value. One correlation id covers a whole action, one parent_call_id covers the calls a single step made, one attempt distinguishes rows that are otherwise identical, and one call_id covers exactly one row. Grain is what makes the four irreducible to each other. A coarser field cannot be recovered from a finer one, because the finer field says nothing about which of its values belong together. A finer field cannot be recovered from a coarser one, because the coarser field has already thrown the distinction away.

parent_call_id is the field people expect to be derivable, and it is not. The calling frame knows which call it is inside; the row does not, unless the frame writes it down at the moment it makes the child call. Reconstructing it afterwards means inferring a tree from timestamps, which is the same inference the time window made, run at a finer grain where it holds up less well rather than better. The sub-agent calls at 09:14:05.1 and 09:14:05.2 share a parent, and the call at 09:14:08.7 has a different one; nothing in the timestamps distinguishes those two situations.

attempt is the field people collapse. A rejected call and the retry that followed it are two billable events carrying the same correlation id, the same parent, the same model and nearly the same timestamp, and without an attempt number the most a query can honestly say is that there were two calls. With it, everything beyond a first attempt is a where attempt > 1, which is the difference between measuring what retries cost and arguing about it.

With those four on the row, fourteen rows become one number and, more usefully, a breakdown of where that number went:

SELECT coalesce(parent_call_id::text, 'root') AS step,
       count(*)                               AS calls,
       sum(cost_usd)                          AS usd,
       round(100 * sum(cost_usd) / sum(sum(cost_usd)) OVER (), 1) AS pct
FROM   llm_calls
WHERE  correlation_id = 'r_7f31a2'
GROUP  BY 1
ORDER  BY usd DESC;

An action is not a conversation#

A correlation id identifies one user action, and a chat product's natural unit is a conversation containing many of them. Conflating the two is the most common way this schema goes wrong after it is otherwise correct, because both are plausibly "the request". A conversation running to 200 turns accumulates hundreds of calls under one identifier, and a GROUP BY correlation_id over that data produces a distribution with a long, meaningless tail — the expensive groups are the long conversations, which everyone already knew. The per-action questions become unanswerable at the same time: whether turn 180 costs more than turn 3 is the single most important economic fact about a chat product with prompt caching and a growing context, and collapsing turns into one group hides it completely.

What actually separates the two is that a conversation is a container and an action is a cause. The conversation existed before the turn and goes on existing after it, and it did not make anything happen. The turn is something a person did, and every call underneath it happened because they did it. Nearly every question anyone asks about spend is causal in that sense — what did this feature cost, what would removing this step save — and a container has nothing to say about any of them.

The resolution is a third identifier with its own grain. session_id for the conversation, correlation_id for one turn or action within it, call_id for one model call. Costs roll up cleanly through all three, and measuring turn cost against turn index — the plot that says whether context growth or compaction dominates your economics — cannot be drawn from a schema that only knows about conversations.

Fan-out that happens on the provider's side#

The fan-out in the trace above was yours: your code spawned the sub-agents, so your wrapper saw every call. Increasingly some of it is not yours. Server-side tools run a sampling loop on the provider's infrastructure, and a hosted agent runtime runs the whole loop there, so a single call from your process can correspond to many model requests you never issue.

The instrumentation still exists and lives in a different place. Anthropic's Managed Agents emit a span.model_request_end event per model request carrying a model_usage object with the four token counters, so a client reading the session's event stream can write one cost row per underlying request with the session's correlation id attached. The session object also carries cumulative usage including a list_cost, and in a multi-agent session each thread carries its own. One property of those figures is worth knowing before building on them: per-thread costs do not sum to the session total, because the session figure additionally includes running time and each figure is rounded independently, so the session-level number is authoritative and the per-thread numbers are for apportionment rather than for totalling. The practical consequence is that a correlation id has to be attached where you create the unit of provider-side work — the session, the batch request, the agent run — in whatever metadata field that surface offers, because it is the only handle you will have when the usage events come back.

That is the mint-upstream chain again, with its first link now sitting outside your own process rather than at your entry point. What comes back afterwards is keyed to the session, so a session created without an identifier does not produce requests that are merely hard to attribute; it produces requests for which no artefact anywhere can be joined to a user action.

The instruction not to total the per-thread figures is worth understanding rather than obeying, because it generalises. Two separate things stop the parts adding to the whole. Each figure is rounded on its own before it reaches you, so a sum of parts accumulates a rounding error the whole never had, and the error grows with the number of parts. And the session figure measures something the threads do not, since running time is a property of the session rather than of any thread inside it, which means there is no thread it could be charged to. A total assembled from parts is therefore short by the time and off by the rounding, and no amount of care in the addition repairs either.

Traces are sampled and a ledger cannot be#

A correlation id is not a trace id borrowed from an observability stack, even though the two carry the same value in a well-plumbed system. Traces are sampled — 1% to 10% head sampling is ordinary, and tail sampling keeps the slow and the failed while discarding the routine — and a cost ledger cannot be, because the 99% you discarded is most of the money. Copy the trace id onto the cost row where one exists, so a spend anomaly can be opened in the tracing UI when the trace happens to have been kept, and do not make the cost row's existence depend on that. Storing cost rows in the tracing backend fails for the same reason from the other end: retention there is typically seven to thirty days, and finance will ask about a quarter.

Sampling is defensible for latency and indefensible for money because the two questions have different shapes. A percentile is a property of a distribution, and a random sample of a distribution is a smaller copy of the same distribution, so keeping 1% of traces answers "what is the slow tail doing" about as well as keeping all of them. A total is not a property of a distribution. It is the distribution multiplied by how many members there were, and the multiplication amplifies whatever the sample got wrong — in particular, whatever it got wrong about the largest members. Model spend has exactly the shape that punishes this. Two rows out of the fourteen above carry 76% of the money, and a sampler keeping one trace in a hundred is as likely to discard those two as any others.

Tail sampling makes this worse rather than better, which is the part that surprises people who reach for it as the fix. Keeping the slow and the failed is a deliberately biased sample, chosen because bias in that direction is useful when debugging, and the same bias runs straight through cost: slow calls are long calls and long calls are expensive. A tail-sampled store therefore over-represents precisely the rows a spend figure is most sensitive to. It is the right sample for an investigation and the wrong one for a sum, and one store cannot be both at once.

The identifier is therefore the cheapest piece of plumbing in this topic and the one with the widest reach, because it converts a table of individually correct rows into a table that can answer questions about units people recognise — a click, a turn, a job. It costs one value generated at each entry point, one field in each queue payload, and a rule that consumers re-enter rather than re-generate.

What it does not do is make any request's cost correct. It makes fourteen rows into one group, and a group is only as good as its members. Every row in that group carries token counts, and the arithmetic that turns those counts into money assumes they are the right counts to multiply. One total_tokens number, however carefully priced and however cleanly grouped, hides two entirely different classes of cost regression behind a single total.