Deciding whether to enable sampling

Two things make this a decision rather than a default. The capability declaration is per connection, so enabling sampling for one server does not enable it for another, and the specification assigns the policy to the client, which means a client that declines a given server's requests is behaving exactly as the specification anticipates. Nothing about declining is non-standard and the servers continue to work.

What makes it hard is that the arguments arrive one integration at a time, from authors with reasonable requests, and "no" without a measurement is a position that erodes. Three arms are worth building for any server that asks. Arm A routes the model call through the client with sampling. Arm B has the server make the call with its own provider key. Arm C has the application make the call itself and pass the result to the server as ordinary tool arguments, never involving the server in the model call at all.

What a fixed workload showed#

The workload has to be fixed before anything runs, or the comparison becomes an argument about which examples were chosen. For a document-retrieval server whose tool returns whole contracts, sometimes 40 pages, that meant 500 real contracts drawn from a fortnight of production traffic, mean 9,200 input tokens. For a routing server with a taxonomy of 240 support categories that it curates, it meant 400 support messages with human-assigned labels held back from the taxonomy's own training examples. Every arm ran the same prompts where the prompts were the server's, and the whole exercise took a week of one engineer's time.

Summarisation, 500 contracts, rated by three reviewers on a 100-document sample:

Classification, 400 human-labelled support messages:

The quality column is the one that surprises people, and its lesson is about arm C rather than arm A. The server authors write better prompts for their own domain than a platform team does — by two and a half to three points of classification accuracy and about a third of a rating point on summarisation — because they have iterated against their own taxonomy for a year. Arm C's deficit is real and it is closeable with effort that would have to be spent repeatedly, once per integration, by people who do not own the domain.

Two of those gaps deserve separating, because a workload of this size can establish one of them and not the other. Arms A and B differ by one tenth of a point on a five-point scale, judged by three reviewers over a hundred documents, and a gap that small is what two reviewers disagreeing about a handful of contracts looks like; the honest reading is that the workload found no difference between them at all. The gap to arm C is a different kind of number, because it turns up in both experiments on two instruments that share nothing — a human rating scale applied to contracts, and agreement with held-back human labels on support messages. Its size alone would not carry it: two and a half to three points of accuracy across four hundred messages is ten to twelve messages changing hands. What makes it worth acting on is that a second workload, measured a different way, moved in the same direction, and that is the property a single larger number would not have had.

Latency shows the extra hop honestly. Arm A adds roughly 300 ms at p50 and 1.6 s at p95 over arm B, which is the request travelling server to client, the authorisation and context resolution running, the provider call, and the result travelling back — plus, at p95, queueing behind the per-server rate limit. That is a small penalty and it is a penalty rather than a benefit, which is worth naming because sampling is sometimes discussed as though mediation made things faster.

The privacy row is where expectations are most often wrong in the other direction. Both servers already possess the data in question: the retrieval server produced the contract, and the routing server receives the user's message as tool arguments. With the context ceiling set to none, sampling does not disclose to either of them anything they did not already have, so the alarm that "the server sees your data" does not apply to these two cases, and applying it anyway would have been an argument that collapses under questioning. What sampling adds here is the prompt authorship row and the budget rows, and those are enough on their own.

The cases separate on who is exposed to the loop#

Read the last three rows of either table together and the decision resolves. Arm A moves $14–15 per 500 contracts from the integrator's account to yours, adds a party that writes prompts against your model, and makes the worst-case spend a function of code you do not control and cannot review at version-bump time. During the trial that stopped being theoretical: eleven contracts were scanned PDFs whose extracted text was garbage, the retrieval server's validation rejected each summary, and its retry loop produced up to fourteen sampling calls for a single document, $0.43 where the median document cost $0.031. On a fleet, with a bad batch of scans, that is the mechanism from chapter 5 — a sampling call nested inside somebody else's retry — with a different server's name on it.

Those figures come from opposite ends of the same table and meet, which is what makes them worth trusting. Fifteen dollars fifty spread across five hundred contracts is a fraction over three cents a contract, and a fraction over three cents is the median document cost quoted above — one arrived at by dividing a total by a count, the other by pricing a single document. Fourteen calls on one scanned PDF at three cents each is forty-three cents, which is the other figure in that sentence. The retry loop is doing nothing exotic, then: it is the same unit price multiplied by a count the client does not set, cannot see in advance, and does not learn about until an invoice arrives. The monthly figure quoted later in this chapter is the same unit again — roughly $2,600 a month at three cents a contract is somewhere above eighty thousand contracts, which is what "one fleet's volume" means once it is written in documents rather than dollars.

A retrieval server of that shape does not need any of this. It already calls three external APIs, it already has a provider account for embeddings, and adding one model call to its own stack is an afternoon's work that leaves the cost, the retries and the prompt iteration with the person who wrote them. Enabling sampling for it buys the integrator a small convenience and buys you a trust boundary, an ongoing bill, and a dependency on their retry logic.

A routing server is a closer call and it turns on deployment shape rather than on use case. Its hosted build has a provider account and falls in the same place as the retrieval server. Its stdio build, which runs as a local process on a support agent's laptop for on-premise customers, has no provider account, no credential to distribute, and in some customer environments no egress to a model API at all. For that build, sampling is the only mechanism that makes the feature exist, and the argument from chapter 1 — that the genuine economics of sampling are keylessness, no egress and no provider configuration — is exactly the argument that applies. That is the case worth supporting, and the policy can say so at the granularity of the transport:

servers:
  routing-server:
    allow_sampling: true
    transports: [stdio]           # the hosted build gets nothing; it has a key
    consent: per_session
    models: [claude-haiku-4-5]
    max_context_scope: none       # it classifies its own tool arguments
    max_input_tokens: 4000
    max_output_tokens: 128        # a label, not an essay
    rate_limit: { requests_per_minute: 30, burst: 60 }
    budget: { window: 24h, usd: 2.00 }   # ~2,200 classifications per user per day
    deadline_ms: 8000
    breaker: { denials: 25, window: 10m }

  retrieval-server:
    allow_sampling: false
    note: "Declined 2026-03. Has its own provider account; measured quality
           difference vs its own call was 0.1/5 on a 500-contract workload."

The comment on the budget line is a derivation rather than a guess, and it is the kind worth redoing before copying a policy block to a second server. Thirty-six cents covered four hundred classifications in the trial, which is a shade under a tenth of a cent each, so two dollars buys something over two thousand of them in a day. Setting that against what a person does gives the budget its meaning: a support agent classifying two thousand messages in a day is classifying one every fifteen seconds for eight hours without stopping, so the ceiling sits far enough above real use that no user will ever meet it. Set against what a loop does, the same two dollars is exhausted in a little over an hour at the thirty-a-minute rate limit. A budget worth having has both of those properties, and neither is visible from the number itself.

Answering the product manager without invoking security#

The question that arrives after the policy file is why the team is making a standard feature hard, and the answer that works is arithmetic rather than principle. Enabling sampling for the retrieval server moves about $15 per 500 contracts onto your provider account, which at one fleet's volume is roughly $2,600 a month, in exchange for a measured quality difference of one tenth of a point on a five-point scale and 300 ms of additional latency. The engineering to make it safe to run — the per-server policy, the error vocabulary, the decision store, the records and quotas, the adversarial fixtures, the fallback paths — is the several weeks the preceding chapters describe, and it is not optional once the capability is declared, because every server that receives the declaration can use it.

The compliance point is short and worth making plainly. The specification defines the mechanism and assigns the policy to the client, recommending human approval and rate limiting without mandating a mechanism, so declining a given server's sampling requests is compliant behaviour. A well-written server degrades on the -32001 frame the way chapter 4 described, and one that does not is telling you something useful about how it will behave in every other failure.

The asymmetry of the two mistakes is the last piece. Enabling sampling later for a server that has proved it needs it costs a policy block and a review. Disabling it after two integrators have shipped features that depend on it is a breaking change to somebody else's product, negotiated under pressure, usually resolved by leaving it on. Starting from off and moving individual servers to on is reversible in the direction you are likely to need.

Turning it on for the one server that earned it#

The rollout is short because the machinery already exists. The capability declaration goes out behind a flag that is on for internal users first, with the routing server the only entry in the allowlist and every other server explicitly allow_sampling: false. The adversarial fixtures from chapter 6 run in continuous integration on every client build, including the reconnect case for the breaker. The records land in the same store as everything else from day one, because a week of traffic with no telemetry is a week you cannot reason about afterwards.

In the first week the numbers to watch are per-server and specific: requests per session against the expected one per classified message, the ratio of contextRequested to contextGranted staying at none-to-none, representedCount above one as a sign of a retry loop, p95 latency against the 8-second deadline, spend against the $2 daily budget per user, and the count of consent grants, since a support agent granting a raise five times a day means the budget is wrong rather than the agent is careless.

The comparison is what makes any of that defensible, and its most useful property is that it can come out either way. Sampling won nothing on quality against a server calling its own model, lost 300 ms, and moved a bill; it won decisively for the one build that has no key, no egress and no way to be configured with either. Deployment shape, not use case, is what separated them, and no amount of reasoning about the mechanism would have produced that answer without a fixed workload behind it.

A revisit date on the calendar for 30 days out, with those six numbers in front of the people who set the policy, is what keeps the decision a decision rather than a default that has quietly become permanent.