Idempotency keys for tool calls

The idempotent hint says that repeating a call with the same arguments has no additional effect. A host reading it treats a timeout as safe to retry, because the annotation told it so. That is the field working exactly as designed, and it is why a false one is more dangerous than no annotation at all: the pessimistic default is "not safe to retry", so setting the flag is an active removal of caution.

That comparison is worth being exact about, because "a wrong hint is worse than no hint" sounds like rhetoric and is really arithmetic about defaults. An absent idempotentHint reads as false, and a host reading false does not retry, so an unannotated tool receives the cautious treatment without anybody having to ask for it. Setting the field to true is the only way to move a host off that behaviour. So from the host's side the field has no neutral position: omitting it and setting it to false produce identical conduct, and only true produces different conduct. A false claim of idempotency is therefore not a missing safeguard. It is an instruction to remove one that was already in place.

The instinctive response to a duplicate is to set idempotentHint: false and consider the matter closed. That changes one host's retry decision and leaves every other way a second identical call can arrive. The property the flag asserted has to become true rather than being withdrawn, because the underlying situation does not go away — a call whose outcome is unknown has to be resolvable, and the only resolution that does not require a human is to make repeating it safe.

An idempotency key is how that is done: a string that names one operation, recorded by the server before the side effect happens, so that a later call carrying the same name is recognised as a repeat rather than performed again. The property it buys is narrow and exact — this operation, from this caller, with these arguments, within this window — and it is the only property a retry actually needs.

Where a repeated call comes from#

Six of those seven rows are unaffected by anything in the listing, which puts the annotation in proportion. It is one input to one class of caller. A second charge could arrive from any of the other rows on any other day, so a fix that only addresses the first row leaves the defect live.

The column that matters is the middle one, and it is worth reading down rather than across. Every row where the answer is no is a row whose cause sits somewhere your listing cannot reach: in another vendor's retry policy, in the transport, in the model's own reasoning, in a second host you have never seen, or in a tenacity decorator on your own client. The listing is a document your server publishes once per session, and a document can only influence a reader who reads it and chooses to act. So the annotation is an input to exactly one row, and the property you actually need — that a second identical call has no second effect — has to hold regardless of which row produced the call. That is a property of the handler, because the handler is the one place every one of the seven paths arrives at.

A key the server derives, not one the model supplies#

Idempotency keys are older than any of this. Payment APIs settled on them in the mid-2010s — Stripe's Idempotency-Key header is the reference implementation most engineers have met — precisely because network timeouts made "did my charge go through" unanswerable, and the fix was to let the caller name the operation so the server could recognise a repeat of it. MCP inherits the problem in a sharper form, because the caller is a model that may not remember what it sent and a host that may replay without asking anyone.

That sharpness rules out the obvious port of the pattern, which is to add an idempotency_key property to the input schema and let the model fill it in. A model generating a fresh UUID on each attempt defeats the mechanism silently, and a model reusing one key across two genuinely different charges suppresses a payment the customer wanted. The key has to be derived by the server from things the server can see, which for a tool call means the authenticated caller, the tool name, and the canonicalised arguments.

Those two failure directions are worth separating, because they call for opposite corrections and a model-supplied key is exposed to both. A fresh identifier on every attempt gives a retry of one operation a different key, so the server sees two operations and performs both — the mechanism is present, running, and doing nothing, which is the worse of the two because it looks installed. One identifier reused across two operations the user genuinely wanted gives two operations one key, so the second is suppressed and a payment somebody asked for never happens. The first is invisible until a customer is charged twice and the second is invisible until a customer complains that nothing occurred. Neither is carelessness on the model's part. It is being asked to hold a piece of protocol state across turns, which is not something its position in the system allows it to guarantee.

import hashlib, json

def idempotency_key(caller: Caller, tool: str, arguments: dict, window_s: int = 900) -> str:
    canonical = json.dumps(arguments, sort_keys=True, separators=(",", ":"))
    bucket = int(time.time()) // window_s          # coarse time bucket
    material = f"{caller.tenant}|{caller.subject}|{tool}|{canonical}|{bucket}"
    return hashlib.sha256(material.encode()).hexdigest()

Canonicalisation is what makes this work at all: {"amount_cents": 24000, "card": "c_71"} and {"card": "c_71", "amount_cents": 24000} must produce one key, and they do once the JSON is sorted and whitespace is fixed.

The step underneath that is worth stating, because it is why the sorting is not cosmetic. A hash is computed over bytes. Those two objects are the same object to any JSON reader, since a JSON object has no defined property order, but they are different byte strings — so hashing them as they arrive produces two keys for one operation, and the mechanism fails open without complaining. Sorting the keys and fixing the separators selects exactly one byte string for each distinct set of arguments, which makes the mapping from arguments to key a function rather than a coincidence of how the caller happened to serialise. Anything else that can vary between two encodings of the same call — whitespace, numeric formatting, optional fields present with default values — has to be pinned for the same reason.

The time bucket is the deliberate compromise. Without it, a customer who genuinely wants to buy the same thing twice in a week is blocked forever; with a 15-minute bucket, a retry seconds later collides and a repeat purchase the next day does not. Where the domain gives you a real business key — an order id, a subscription period, an invoice number — use that instead of the bucket, because it expresses the actual uniqueness rule rather than approximating it with time.

The bucket's width decides which of two mistakes you make, and the code above has already chosen: the fifteen-minute window is the nine hundred seconds in the signature. Narrow it and a retry arriving after a slow upstream can fall into the following bucket, produce a different key, and not be recognised as a repeat — the mechanism stops working precisely when it was needed. Widen it and a customer who legitimately wants to buy the same thing twice inside the window is refused a purchase they meant to make. Neither error announces itself, which is why a genuine business key is better wherever the domain supplies one: an order id or a subscription period states the uniqueness rule directly instead of approximating it with a clock.

The record has states, and the middle one is the hard one#

Storing the key is not enough; what makes the second call safe is that the first call's record exists before the side effect, and carries the outcome afterwards.

The self-loop on Claimed is where duplicate-suppression schemes usually break. A retry that arrives while the first attempt is still talking to the processor finds a row that exists but has no answer yet, and both "wait for it" and "treat it as absent" are wrong — the first blocks a connection for as long as the upstream is slow, the second is exactly the duplicate you were preventing. Reporting that the operation is in flight, with the key, and telling the caller to check rather than re-submit is the honest third option.

Both rejected options fail for the same underlying reason, which is that the record exists and the outcome does not. Waiting for the first attempt makes the second caller's latency a copy of the first's, so a slow processor now occupies two connections where it occupied one, and under load that is the shape by which a queue becomes an outage. Treating the row as absent discards the only evidence that a first attempt exists, which is exactly the situation the row was inserted to prevent. The third option declines to answer a question nobody can answer yet and says so plainly, which costs the caller a round trip and a follow-up call, and costs nobody a second charge.

async def charge_card(caller: Caller, arguments: dict) -> CallToolResult:
    key = idempotency_key(caller, "charge_card", arguments)

    row = await db.fetchrow(
        """INSERT INTO idempotency (key, tenant, status, created_at)
           VALUES ($1, $2, 'in_progress', now())
           ON CONFLICT (key) DO NOTHING
           RETURNING key""",
        key, caller.tenant,
    )
    if row is None:                                   # somebody got here first
        prior = await db.fetchrow("SELECT status, response FROM idempotency WHERE key=$1", key)
        if prior["status"] == "completed":
            return ok_result(prior["response"])       # same answer, no second charge
        return error_result(
            f"A charge with these exact details is already in progress (ref {key[:12]}). "
            f"Do not submit it again; call get_charge_status with this ref."
        )

    try:
        receipt = await processor.charge(
            card=arguments["card"], amount_cents=arguments["amount_cents"], reference=key,
        )
    except ProcessorTimeout:
        return error_result(
            f"The processor did not answer within 30 s. The charge may or may not have "
            f"been taken; ref {key[:12]} is recorded. Call get_charge_status with this "
            f"ref before doing anything else. Do not retry this call."
        )
    await db.execute(
        "UPDATE idempotency SET status='completed', response=$2 WHERE key=$1",
        key, receipt.as_json(),
    )
    return ok_result(receipt.as_json())

Two details in there are load-bearing beyond the deduplication. The key is passed to the processor as its own idempotency reference, so the guarantee extends across the boundary your server does not own — without that, your row says in_progress while the processor happily takes a second authorisation from a retry your server never saw. And the timeout branch does not lie about what happened, in the terms the previous chapter set out: the outcome is genuinely unknown, the result says so, and it names the tool that can resolve it. The cost of the whole mechanism is one indexed insert, measured at 1.8 ms median against a Postgres instance in the same region, and about 400 bytes per row for a table that a nightly job trims to 48 hours.

Set those figures against the thing they prevent and the trade stops being a judgement call. Under two milliseconds is added to a call already spending the better part of a second talking to a payment processor, so the latency is a rounding error on the operation it protects. Four hundred bytes a row kept for two days means the table's size is bounded by two days of traffic rather than by the age of the service, which is the property that stops it becoming somebody's problem a year from now. On the other side of the comparison is one duplicate charge, which costs a refund, a support conversation and a customer's confidence. There is no traffic volume at which that arithmetic reverses, which is worth knowing in advance, because the objection this proposal draws is nearly always about the write.

The limits of an argument hash#

A derived key does not make the tool idempotent in the general sense. It makes this operation, identified by these arguments from this caller within this window, safe to repeat, which is narrower and is the property that matters for retries. Two charges with amounts differing by a penny produce two keys and two charges, correctly, and no scheme built on argument hashing can tell an intentional second purchase from a mistaken one.

It also does not extend to anything the tool did outside the transaction. If the handler charges the card and then sends a receipt email, a replay that returns the stored response sends no second email — which is right — but a crash between the charge and the email leaves the record completed with no email ever sent, so the follow-on work belongs in a queue keyed by the same reference rather than inline. And it is not a substitute for a reconciliation job: keys prevent duplicates within their window, and the row that says in_progress forever because the process died is a real state that something has to sweep.

The test that keeps the property true is short and belongs in the same suite as the refusal tests from the fourth chapter of this topic: fire the same tool call twice concurrently against a stubbed processor that sleeps for two seconds, assert the stub was invoked exactly once, and assert that the second call came back with an in-flight message rather than a receipt. Then run the pair sequentially and assert the second returns the stored response. A carrier or processor stub that counts invocations is the whole harness, and the counter is the assertion that no amount of reading the handler can replace.

The counter is doing something a more obvious assertion would not, which is worth seeing before somebody replaces it with an easier check. Asserting that both calls returned a receipt passes whether or not the processor was called twice, because a duplicate charge also returns a receipt — the observable the caller sees is identical in the correct and the broken case. The only place the two differ is on the far side of the boundary, at the processor, which is why the stub has to be the thing that records what happened. That is also why the concurrent case needs the sleep: without it the first call finishes before the second begins, the second finds a completed row, and the self-loop on Claimed — the branch that is hardest to get right — is never exercised at all.

The same reasoning settles the question for a create_shipment tool that occasionally times out against a carrier's API. Asked to defend or remove its idempotent annotation, the answer depends entirely on whether a shipment created for the same order, address and service level within the window is recognised as a repeat. If the handler simply POSTs to the carrier each time it is called, the flag is false and is actively causing the duplicates support is reporting, so it comes off — and taking it off fixes one row of the table above and leaves the rest. If instead the handler derives a key from the order id, records it before the carrier call, and passes it as the carrier's own reference, then the flag is true, it stays, and a host that retries on the strength of it gets the same tracking number back.

The flag follows the code; it never leads it. That is the rule the whole subject reduces to, and it generalises to every behavioural annotation: a hint is documentation of a property, not a request for one, and the review question for any of them is which lines of the handler make the claim true. If nobody can point at those lines, the annotation is a description of an intention, and hosts do not read intentions.

Everything in this topic so far has been about a tool that gets called and what happens when it does. The failure that consumes the most engineering time in practice is quieter: a tool that is declared correctly, listed correctly, annotated honestly, and never chosen at all.