An abstention threshold is a two-sided cost, so measure false abstentions

A false abstention is a question the system refused that it would have answered correctly. It is the other half of the cost of a gate, and it is the half nothing observes on its own. A wrong answer that reaches a user has a path to a complaint queue: the user notices, the user is annoyed, the user says so, and a count goes up. A refusal produces nothing on that path at all — the user reads a decline, forms whatever impression they form, and leaves no record of whether an answer was available.

That asymmetry is not a gap in anybody's instrumentation discipline; it is a property of the two events. One of them creates a dissatisfied person holding evidence, and the other creates an absence. Any process that tunes a threshold by watching the first and not the second is watching a quantity that moves in one direction as the threshold rises, which makes it useless as a stopping rule, and the rest of this chapter is about building the other half.

A monitor that can only turn one way#

The dotted edge is the defect, and the chain that makes it fatal is worth stepping through link by link, because each link is obviously true and the conclusion is not obvious at all. Raising the threshold reduces the number of answers served, since fewer scores clear a higher bar. Fewer answers served means fewer wrong answers reach users, since a wrong answer must be served to reach anyone. Complaints are generated only by served answers. Therefore raising the threshold always reduces complaints — not usually, and not on average, but as a matter of what complaints are counted over. Therefore the sensor cannot distinguish a move that removed mostly wrong answers from one that removed mostly right ones, because both register as a drop, and the cost the second one incurs never reaches the comparator at all. Therefore the loop has no point at which it reports a problem.

Follow it to the end and the terminal state is a system that refuses nearly everything and has a perfect complaint record. A team running this loop honestly and attentively will move the threshold up, then up again when the count falls further, and will keep going while every number on the dashboard improves. Nothing along the way misbehaves. This is the strongest form the stumble takes in this topic: the complaint count is a quantity that improves monotonically while the dangerous property — the correct answers being thrown away — varies freely and is never observed at all.

Measuring the side that is silent#

The quantity missing from the loop is the false-abstention rate, and the measurement is unglamorous rather than difficult.

Sample refused questions from real production traffic — refusals from a benchmark are not the same population, because production questions are phrased worse and skew toward a corpus's weak spots. Run each one through the pipeline with the gate disabled and log the answer that would have been shown, which is a shadow evaluation and reaches no user. Then grade those answers the same way the original evaluation set was graded.

Two details in that procedure carry most of its validity, and both are causal claims rather than hygiene. The gate-disabled run has to use the same retrieval results the original request got, which means logging retrieved passage IDs at request time rather than re-retrieving during the shadow run. Re-retrieving against a corpus that has grown since produces a different and generally better answer, so more of the refused questions come back correct than would have at the time, and the measurement will understate the false-abstention rate. And the graders must not see the gate score, since a grader who knows the system refused an answer marks it harshly, so more of the shadow answers are graded wrong than deserve it, and again the measured rate is too low. Both biases push the same way. Two independent errors pointing in the same direction do not cancel, they compound, and the direction they point is toward concluding that refusals are cheap.

On 400 refusals sampled from one month's traffic and re-run at a hypothetical 0.85 cutoff, 34% were answers that graded correct. That figure carries a 95% interval of roughly ±4.6 points, which is tight enough to distinguish 0.85 from 0.9 and not tight enough to distinguish 0.85 from 0.87 — the second pair sits inside the interval and the first does not. Sizing the sample against the decision you are making avoids grading four hundred answers to resolve a difference the sample cannot see.

def operating_points(scores, correct, cutoffs):
    """scores: gate score per question; correct: grade of the answer that would be served."""
    n = len(scores)
    rows = []
    for t in cutoffs:
        served   = scores >= t
        refused  = ~served
        rows.append({
            "cutoff": t,
            "abstention_rate": refused.mean(),
            # of the questions we refuse, how many would have been right
            "false_abstention_rate": correct[refused].mean() if refused.any() else 0.0,
            # per 1,000 questions
            "correct_withheld": 1000 * (refused & correct).mean(),
            "wrong_served":     1000 * (served & ~correct).mean(),
        })
    return rows

Run across the entailment score on 2,000 graded answers, the table is the deliverable of this chapter.

Three of those six columns are derived from the others, and checking that is what makes the table safe to reason from.

The fourth column is the product of the two before it. The code says so — 1000 * (refused & correct).mean() is the share refused times the share of those that were correct, scaled to a thousand — and every row obeys it. Seven percent of a thousand questions is seventy refusals, and twenty-one percent of seventy is about fifteen. Forty-one percent of a thousand is four hundred and ten, and forty-three percent of that is a little over one hundred and seventy-six. The same multiplication reproduces twenty-nine and thirty-four percent as ninety-nine, and sixty-three and fifty-eight percent as three hundred and sixty-five. So the column adds no fourth measurement: the only quantities genuinely measured here are the abstention rate, which comes from the score distribution, and the false-abstention rate, which comes from the graded shadow run.

The first row's remaining figure closes against chapter 1. Two hundred and ninety wrong answers served per thousand, with no gate at all, is the complement of an accuracy of seventy-one percent — the figure from the graded set this whole topic has been measuring against. Two chapters, one number, arrived at from opposite ends.

The last column closes as differences of the two before it, and it is the number a review meeting needs. It is the ratio of correct answers newly withheld to wrong answers newly suppressed, moving from the row above to the row in question. Going from no gate to 0.5 withholds fifteen correct answers and removes seventy-four wrong ones — two hundred and ninety minus two hundred and sixteen — which is roughly one correct answer for every five wrong ones, close to free. Going from 0.85 to 0.9 withholds 77 correct answers, since one hundred and seventy-six minus ninety-nine is seventy-seven, to remove 37 wrong ones, since eighty-eight minus fifty-one is thirty-seven, and that ratio is the 2.1 in the table. Going from 0.9 to 0.95 withholds 189 to remove 33, which is the 5.7. The exchange rate degrades monotonically, and because it is the slope of a table anyone can read rather than a fifth measurement, nobody has to trust it separately from the rows. A move from 0.7 to 0.85 sits at the point where it crosses one to one.

One column does not close, and saying so is more useful than papering over it. The wrong-answers-served column cannot be reconstructed from the abstention and false-abstention rates against the two hundred and ninety of the first row; doing that multiplication gives systematically larger values on every row. The two halves of the table come from two instruments — the abstention and false-abstention figures rest partly on the hand-graded sample of four hundred refusals, and the served-answer figures on the two-thousand-answer graded set — so they are two measurements of overlapping populations rather than one. The withheld column closes because it is internal to one of those instruments. The served column does not, and no derivation below spans it.

Whose number this is#

An engineer cannot decide where on that table to sit, because the decision requires knowing what a wrong answer costs relative to a refusal, and that is a property of the business and the domain rather than of the model. What an engineer can do is put the question in a form somebody can answer. Asking a support lead to price a wrong answer in euros produces a shrug, because a euro figure requires them to invent a scale. Asking them to choose between adjacent rows produces a number, because the choice is concrete and both options are in front of them.

The usable form is a bisection on the exchange rate. Would you accept 10 extra refusals to remove one wrong answer? — for one internal support assistant the answer was yes. A hundred? — no. Thirty? — hesitation, then no. The lead settled at roughly 3:1, which puts the operating point between 0.7 and 0.85. The same conversation with a clinical safety lead about protocol lookups produced 40:1, which pushes past 0.95 and comes with a second decision about what the refusal message says and where it sends the clinician, because at that operating point the refusal path is the main product surface rather than an edge case.

The binary framing also hides an option that changes the exchange rate rather than moving along it. Between answering and refusing there is a band — say 0.4 to 0.7 on the entailment score — where the system can serve the answer with the supporting passages shown and the unsupported claim marked, which gives the user the material to judge for themselves instead of either asserting or withholding. On one support assistant, routing the middle band to that treatment instead of to a flat refusal cut the abstention rate from 29% to 14% at the same wrong-answer rate, because roughly half of what the 0.85 cutoff was refusing was correct and could be verified by the reader in a few seconds. That is a different kind of move: it does not choose a row, it produces a table with better rows, by turning some refusals into cheap verifications. It costs a real product surface and only works where the user can evaluate a passage, which is true of an engineer reading a runbook and much less true of an end customer.

The deliverable to whoever owns the decision is the table with intervals, the exchange rate column, and a description of what a refusal looks like to the user. The engineering position is that the cutoff is a policy parameter with a documented owner, not a constant in a config file that was set once by whoever wrote the gate.

The number moves when the system does#

A threshold is a statement about a score distribution, and it is only valid while that distribution holds. Three things move it, and all three happen routinely.

Each row has a distinct signature, which is what makes the table a diagnostic rather than a warning. A change that raises the abstention rate and leaves the false-abstention rate flat is a change in the score distribution's location; a change that raises both is a change in how well the gate is working; and a change concentrated on two-hop questions points at the split-support failure of chapter 5 rather than at anything about the threshold. So the movement itself tells you which component to look at, provided the false-abstention rate is being re-measured rather than assumed.

The model upgrade case is the one that catches teams, because nothing in the release touched the abstention layer and the abstention rate nearly doubled. Re-running the operating-point table on a fresh graded sample belongs in the release checklist next to the accuracy evaluation, and the graded sample has to be refreshed too, since a question mix from six months ago describes a product nobody is using any more.

Guarantees, population policy, and retrieval in an abstention costume#

A clinical safety lead's question — can you guarantee the system will never answer when it should not — has one honest answer, which is no. Every component in the chain is statistical: the retriever misses passages, the entailment model is 41% wrong on negation, the decomposition step drops qualifiers, and the threshold is fitted to a sample. A guarantee of never answering wrongly is achievable by never answering, and that is the only construction that provides it, which is worth saying plainly because it locates the guarantee in the trivial corner of the table rather than in a component somebody could go and improve.

What can be offered in its place is specific and worth writing down: a measured false-answer rate with a confidence interval, on a named question distribution and a named corpus snapshot; a monitored recheck that reruns the measurement on every model, retriever or corpus change; a per-refusal audit trail showing the claim and the passages it scored against, so a refusal can be explained and disputed; and an escalation path so that a refused clinician reaches a human within a stated time rather than a dead end. Those are commitments a team can keep, and the conversation about whether they are sufficient is the one the safety lead actually needs to have.

Two things this chapter's machinery does not do are worth naming. It is a population-level policy and not a per-question judgement, so it will refuse a specific question that a specific expert knows to be answerable, and no amount of tuning removes that — a threshold sorts by a score and cannot make an exception it has no representation for. The audit trail exists so that case can be argued rather than absorbed.

And a high abstention rate is often a retrieval problem wearing an abstention costume. In one runbook corpus, 26% of production questions never retrieved a supporting passage, and every one of those becomes a refusal at any sensible cutoff, because a gate that asks whether the passages support the claim will correctly say no when there are no relevant passages. Moving the threshold trades those refusals for wrong answers and cannot convert them into right ones. The fix that would actually move both columns is better chunking and a hybrid retriever, one that runs keyword matching and vector search together so that a question phrased in the user's vocabulary can still find a document written in the company's.

All of this is a substantial amount of apparatus — a graded set, a shadow evaluation, a cost conversation, a release-time recheck — and the first response to a proposal that large is usually a suggestion that there is a cheaper way to get most of it.

There is a sentence people add to the system prompt. It takes ten seconds to deploy, and there is an evaluation that appears to show it works.