Independent uncertainty signals

Agreement fails as a routing signal because the thing doing the flagging is the thing being flagged, working from the same input, with the same beliefs. That diagnosis is also a specification: whatever replaces it has to break one of those three identities. It can read something the generator did not read, it can be a different model with different training, or it can be a mechanism that is not a language model at all.

Independence here is an operational property rather than a statistical one, and the reason it is the property that matters comes out of what a routing signal has to do. A signal ranks cases, and the ranking is useful only where the score runs higher on the cases that are wrong than on the cases that are right. Now follow what happens when the scorer shares the generator's beliefs. The generator is wrong on some case because it misread the instruction or holds a mistaken view of how a policy category works. The scorer, being the same weights reading the same input, holds that same mistaken view. It therefore finds nothing to object to, and reports the case as safe. The cases the ranking pushes furthest towards the bottom are exactly the cases that needed to be at the top.

That is why a dependent signal does not merely fail to help — it is actively anti-correlated on the errors that come from shared belief, and the only thing keeping its capture curve near the diagonal rather than below it is the minority of errors that are not shared. An error that arises from sampling noise, where the model's distribution is genuinely split and one draw happened to land badly, does show up as disagreement. Agreement's lift of 1.4× is bought entirely from that minority. The signal is not broken; it is measuring the small part of the error and blind to the large part, and no threshold rearranges which part is which.

The measurement does not change, which is what makes this a series of afternoons rather than a series of projects. A capture curve takes any per-case score that claims to rank suspect cases above safe ones and asks what fraction of the system's errors sit in the top X% of that ranking. Build it once against a pluggable scoring function and every subsequent candidate is a function to write and a number to read.

def capture_curve(cases, score_fn, wrong_fn, budgets=(0.02, 0.05, 0.08, 0.15)):
    """Fraction of all errors caught, per escalation budget, for any signal."""
    ranked = sorted(cases, key=score_fn, reverse=True)   # most suspect first
    total_errors = sum(wrong_fn(c) for c in cases) or 1
    rows = []
    for b in budgets:
        k = max(1, int(b * len(cases)))
        caught = sum(wrong_fn(c) for c in ranked[:k])
        rows.append({
            "budget": b,
            "errors_caught": caught / total_errors,
            "lift": (caught / total_errors) / b,
        })
    return rows

Where the signal becomes available matters as much as how strong it is. A score computed after generation can only decide what happens to an answer that has already been paid for, while a score computed from the input decides which model gets called at all, which is a different and generally more valuable decision.

A check that consults the record instead of the answer#

Some properties of a claim can be settled without judgement of any kind. The policy number either exists in the policy table or it does not. The line items either sum to the stated total or they differ by an amount you can print. The date of loss either falls inside the coverage period or outside it. The assigned category either appears in the schedule of categories that policy covers or it is absent.

def objective_checks(claim, extracted, policy_db) -> list[str]:
    """Failures here are facts about the record, not opinions about the answer."""
    policy = policy_db.get(extracted.policy_number)
    failures = []
    if policy is None:
        failures.append("policy_not_found")
        return failures                      # nothing else is checkable
    if abs(sum(extracted.line_items) - extracted.total) > 0.01:
        failures.append("line_items_do_not_sum")
    if not (policy.start <= extracted.loss_date <= policy.end):
        failures.append("loss_outside_coverage")
    if extracted.category not in policy.covered_categories:
        failures.append("category_not_covered")
    return failures

Those four fired on 4.1% of traffic. Of the flagged items, 47% turned out to have a materially wrong disposition, against a base error rate of 11.1%, and the flags accounted for 29% of all errors in the month. That is a lift of 7.1× at a budget of 4.1%, and it costs a database round trip and about eight milliseconds. A smaller version of the same effect shows up inside a best-of-N selector, where eliminating candidates on two verifiable claims beat every model-based judge on the same task. Used as a routing signal rather than as a selection signal inside a best-of-N step, the same kind of check is stronger, because a flag does not have to identify the right answer to be worth escalating.

Where the independence in these four checks comes from is worth tracing, because it is not a property of the code being deterministic. The policy table is maintained by an underwriting process that has never seen this model's output and does not know the claim is being read. So whatever can make the table wrong — a record entered late, a coverage period keyed from the wrong form — has no causal connection to whatever can make the extraction wrong, which is a smudged scan, an ambiguous line item, a misread date on a photocopy. Two error sets with no shared cause overlap only by coincidence, which is exactly the condition the takeaway named. The check is a second look at the world rather than a second opinion about the answer.

The two ratios reported above are two views of the same quantity, and a report that gives both is worth reading with a pencil. One view is the flag's precision against the base rate: flagged items are wrong far more often than an average item, which says a flag is evidence. The other is capture against budget: 29% of the month's errors sat inside 4.1% of traffic, and 29 divided by 4.1 is the 7.1× quoted. The two should land in roughly the same place, because concentrating errors and being precise about them are the same fact stated from opposite ends, and where they do not, the flagged population and the error population have usually been counted on slightly different definitions. Checking that they reconcile is free and it is the first thing to do with any verifier's report, including your own.

The limitation is coverage, and it is severe enough that a verifier can never be the whole design. These four checks say nothing about whether a claim was correctly categorised as a total loss, whether the described damage is consistent with the photographs, or whether the adjuster's reserve is reasonable — which is most of the judgement the service exists to make. A verifier catches the errors that intersect its checkable surface and is blind everywhere else, so the 71% of errors it misses are not randomly distributed; they are concentrated exactly in the reasoning the checks cannot reach. A high lift with narrow coverage is a good first filter and a bad sole filter.

There is also a discipline about where the check gets its facts. Asking the model to confirm that the policy number is valid, or to restate whether the totals add up, reproduces the generator's view of the document and inherits its errors. The check has to read the policy table and do the arithmetic itself, or it is not independent of anything.

That shortcut is taken often, and it is taken for a tempting reason. A confirmation call costs one request and no new infrastructure, needs no access to the policy database, and does sometimes come back having changed its mind, which reads as scrutiny doing its job. It is not scrutiny. The second call reads the same document through the same weights, so conditioned on the first pass having misread a figure, the second pass misreads it at close to the same rate. The cases where it does change its mind are concentrated in the dispersion errors that agreement was already catching, so the extra call buys a little of what was cheapest to get and none of what was expensive. The question to ask of any proposed check is not whether it is a separate call. It is whether the information it consults could have been wrong for reasons the generator's answer could not.

Disagreement across families rather than across samples#

Mixing model families takes the probability that two wrong samples share the same wrong answer from 0.34 down to 0.11 on an extraction task. That number is usually collected as an argument about voting, and it has a second use: when errors decorrelate, disagreement becomes informative, because two models failing in the same way at the same time is now genuinely unlikely rather than the default.

The reason that decorrelation converts into a routing signal is a short chain, and it is worth having in full because it explains why the three disagreement-based signals in this chapter rank the way they do. Two calls come back with a disposition each. Where the answers are mutually exclusive labels — one disposition per claim, which is the case here and is not the case for free text — two different answers cannot both be right, so a disagreement is a guarantee that something is wrong somewhere. Disagreement therefore has no false-alarm mode in the sense that matters: it never fires on a case where both answers were fine. What it has instead is a blind spot. The one way for both calls to be wrong and the flag to stay silent is for them to be wrong in the same way, since only then do they agree. The share of joint errors that hides behind agreement is the shared-wrong-answer probability, which is precisely the number that fell from 0.34 to 0.11.

Cutting that number to under a third of its old value cuts the blind spot in the same proportion, and the ordering of the measurements follows directly. Sampling one prompt five times leaves the shared-wrong-answer probability at its highest, because every draw runs through the same weights under the same reading of the instruction, so the blind spot swallows most of the errors and the lift is 1.4×. A paraphrase changes the reading of the instruction while leaving the weights alone, which removes some of the shared cause, and the lift is 3.1×. A second family changes the weights and the training data as well, which removes more of it, and the lift is 4.4×. The three numbers are not three unrelated results. They are one quantity — how much of the cause of error the two calls still have in common — measured at three depths.

Replacing five samples of one model with one call to each of two families, and treating any disagreement on the disposition as an escalation, fired on 9.4% of traffic and captured 41% of all errors — a lift of 4.4×, using two generation calls per request instead of five. Both the cost and the signal moved in the right direction at once, which is unusual enough to be worth stating plainly: the N=5 vote had been paying for the weakest available uncertainty signal at two and a half times the price of a better one.

Prompt paraphrases are the cheaper approximation of the same idea. Two materially different phrasings of the instruction, one call each, disagreeing on 7.0% of traffic, captured 22% of errors for a lift of 3.1×. That is worse than the cross-family version and much better than sampling one prompt, because a paraphrase changes the interpretation of an ambiguous instruction while temperature only perturbs the tokens.

Neither version is free of shared error. Two large instruction-tuned models have overlapping pretraining data and overlapping post-training objectives, so a misconception widely represented on the public internet will appear in both, and a claim that is genuinely ambiguous under the policy wording will confuse both. Cross-family agreement is therefore weaker evidence of correctness than its lift suggests, and the operational cost is real: two integrations, two sets of formatting quirks, two rate cards, and a second vendor whose incidents you now also inherit.

The features you can see before the call#

Every signal so far requires generating an answer first. Historical claims came with 60,000 labelled outcomes and a set of attributes visible at intake — page count, OCR mean confidence, number of vehicles, whether the submission crossed two policy categories, presence of handwritten annotation, channel, claimed amount, days between loss and submission, and three more — and a gradient-boosted model over those eleven features, an ensemble of small decision trees where each successive tree is fitted to the errors the ones before it made, predicted the probability that the pipeline would get the claim wrong.

FEATURES = ("pages", "ocr_conf", "n_vehicles", "spans_categories", "has_handwriting",
            "channel", "amount", "days_to_report", "attachments", "prior_claims", "state")

def risk_score(claim) -> float:
    """Trained on historical (input features -> was the answer wrong) pairs."""
    return MODEL.predict_proba([[getattr(claim, f) for f in FEATURES]])[0][1]

def route(claim, budget_threshold: float) -> str:
    score = risk_score(claim)                 # ~1 ms, no model call
    return "strong_model" if score >= budget_threshold else "cheap_model"

At an 8% budget this captured 38% of errors, a lift of 4.8×, at an inference cost of roughly one millisecond. The property that makes it valuable is not the lift alone but the position: because the score exists before any generation call, the escalation can be to a stronger model rather than to a person, and the cheap path never pays for samples it did not need. Sending 8% of intake to a stronger model at four times the per-call price raised generation spend by 31% rather than by 400%.

The gap between those two figures is a weighted average doing its ordinary work, and it is worth saying which weights. Spend after routing is the cheap price paid on the traffic that stays cheap, plus the strong price paid on the slice that moves. Only the slice that moves pays the extra, so the increase is the escalated share multiplied by how much extra that share now pays — not by the price multiple on its own. A small share at a large multiple therefore lands in the tens of per cent, while sending everything to the strong model multiplies the whole bill by the multiple. The common error is to hear "four times the price" and reach for the second number, which is the price of a decision nobody proposed.

One property of this router deserves its mechanism rather than its assertion, because it is the one thing agreement never had. The router is fitted on 60,000 pairs of the form these input features, and this answer turned out to be wrong, and the fitting procedure's whole objective is to make the predicted probability of being wrong match the rate at which answers with those features actually were wrong. Tracking observed correctness is therefore not a happy property of the output; it is the thing being optimised. Vote share was optimised against nothing. It is a byproduct of sampling that happens to be lying around, and whether it tracks correctness is a question that has to be discovered on labelled data afterwards — which is the calibration check, and on the claims task the answer was no.

Two honest qualifications belong with the router. The model is trained against observed correctness, so it is calibrated by construction in a way agreement never was, and it inherits the standard obligations of a trained component: it drifts when the traffic mix or the generator changes, it needs its labels refreshed, and it must be retrained after every model upgrade or it will keep routing according to last quarter's weaknesses. It also learns whatever is predictive, including proxies you may not want to route on — submission channel correlated with a customer segment on this dataset, and the feature had to be dropped after review rather than because the model performed worse without it.

The slice you already knew about#

The cheapest signal in the set required no model and no measurement beyond one that had already been run. A per-slice breakdown that found families of input whose accuracy sat flat regardless of N is already a routing rule: here it was commercial auto with multiple vehicles, 6.2% of traffic at 68% accuracy against 89% overall. Routing that slice wholesale to review captures 17.9% of all errors at a 6.2% budget, a lift of 2.9×, and the implementation is a predicate on a field already in the request.

That 17.9% is the same multiplication that identifies where errors sit in any bucketed table, run once on a slice instead of on an agreement level. The slice is wrong about a third of the time, since its accuracy is 68%, and it is 6.2% of traffic, so it contributes roughly two wrong answers for every hundred items of traffic that pass through the service. The service overall is wrong about one time in nine, its accuracy being 89%. Dividing the slice's contribution by the whole gives a shade under a fifth of all errors, which is the figure quoted, and dividing that in turn by the 6.2% budget gives the 2.9×.

What that calculation needed is the whole reason the rule costs nothing. It needed a traffic share and an accuracy, per slice — two columns that a per-slice breakdown already prints, produced for some other purpose weeks earlier. No held-out set was labelled for it, no model was trained, and no generation call was added. A routing rule of real value was sitting inside a table nobody had read as a routing rule.

The lift is the lowest of the four and the properties around it are the best. It is stable, because it does not depend on a model's behaviour on the day; it is auditable, because the rule is one line and a regulator can read it; and it degrades gracefully, because when the slice improves the rule simply stops catching much and can be retired. Most usefully, it is available on the afternoon you discover the slice, which is typically a quarter before any trained router is in production.

Catching an error is not repairing it#

A capture curve measures which items reach the escalation path and says nothing about what the escalation path does with them. That second number is where several of these designs lose most of their apparent value. Escalating to a stronger model corrected 51% of the errors that reached it, because a substantial share of the misconceptions were shared across model families in exactly the way described above; human review corrected 88% at roughly twenty times the cost per item. Multiplying through gives errors removed per dollar, which is the figure to compare against the price of an error, and it reorders the options: the objective checks repair 94% of what they catch and are the clear leader on that basis despite their narrow coverage.

"Multiplying through" is two operations and it is worth doing them separately, because each one reorders the table in a different direction. The first is capture times repair rate, which converts a ranking claim into a count of errors that actually go away. That product is what makes lift alone misleading: a signal with a high lift feeding a path that repairs half of what it sees removes no more than a signal with half the lift feeding a path that repairs nearly everything. The second is dividing by what the path costs. Human review repairs 88% against the stronger model's 51%, a bit under twice as much, and costs roughly twenty times as much per item — so on the items a stronger model can reach at all, it is ahead by close to an order of magnitude per dollar spent.

The objective checks lead this table because both operations favour them at once. Their repair rate of 94% is near the maximum available, since a flag that says the totals do not sum routes to a rule-driven correction rather than to another opinion, and their cost is a database round trip. The narrow coverage still binds, and it binds on the numerator: 94% of the 29% of errors they catch is a bit over a quarter of all errors, and no amount of cheapness moves that ceiling. Being the best value per dollar and being sufficient on their own are different claims, and only the first one is supported here.

Verifiers acquire a second failure mode once anything starts optimising against them. If candidate selection or a fine-tuning loop uses the check as its objective, the system learns to satisfy the check, and the correlation between passing and being correct weakens in the direction that hurts most. Keeping a held-out check that is never used for selection, and comparing the two periodically, is the cheap defence.

The general problem underneath all of this is unsolved. There is no inexpensive, well-calibrated, task-agnostic uncertainty estimate for a language model, which is why the working answers here are all task-specific plumbing rather than a method. The nearest thing to a principled tool is conformal prediction, which converts any score — logprobs, a trained verifier, agreement — into a prediction set, a set of candidate labels rather than a single answer, carrying a distribution-free coverage guarantee that the true label sits inside the set at a rate you choose in advance, with no assumption about the shape of the underlying data. It is worth knowing about for tasks with a small discrete label space. The guarantee it provides is marginal rather than conditional, meaning it holds on average across your traffic and not within a slice, so a slice with a displaced answer distribution survives it intact.

The word "marginal" is carrying the whole caveat there, and it is a caveat about averages rather than about the method. The procedure is calibrated on a pool of traffic and promises a coverage rate over that pool. An average can be hit in more than one way: over-covering the bulk of easy inputs by a margin and under-covering a hard slice by a matching margin sums to the same average as covering everything evenly. Nothing in the procedure objects, because the slice was never one of its inputs — it was given scores and outcomes, not a breakdown by commercial auto or scan quality. So the guarantee is real and it is a guarantee about a population you may not be able to act on. Getting a per-slice guarantee means running the calibration per slice, which needs enough labelled data inside each slice to do it, and that is the same constraint that made thin agreement buckets unmeasurable.

Four questions, then, about any proposed uncertainty signal, and they are answerable before it is built. Does it see something the generator did not, or fail by a different mechanism — because if the answer is neither, its capture curve will sit on the diagonal. Where does it run, before the generation call or after, because that decides whether it can route to a better model or only to a queue. What is its lift at the budget you can actually afford, measured on the same curve as everything else. And what fraction of what it catches does the escalation path repair, because a signal with a lift of 7 feeding a path that fixes nothing is worth less than a signal with a lift of 3 feeding one that fixes almost everything.

Everything in this chapter and the four before it has assumed a fixed budget, spent on more calls to the model you already have. There is a different way to spend the same money, which is one call to a better model.