A reranker score orders candidates without claiming any of them are good

A reranker returns a list and a column of numbers beside it. The list is the product of a ranking objective and is the thing the model was trained to get right; the numbers are a by-product, and reading them as a statement about quality is where a retrieval pipeline quietly acquires the ability to hand a generator ten passages that answer nothing. A relevance score here is whatever scalar the second-stage model emits for one query-document pair, and the question this chapter is about is what, if anything, that scalar asserts about the pair on its own.

The practical form of the question is a configuration line. A pipeline that sets top_n to a constant is asserting that every query has that many passages worth showing, and the only evidence available about whether that assertion holds for a given query is the score column it is ignoring. Getting a bar out of that column takes two things the raw number does not supply: a mapping from score to something interpretable, and a downstream contract that can survive receiving fewer passages than it asked for.

The number is a logit, and its zero point is arbitrary#

A cross-encoder ends in a linear layer over the pooled [CLS] representation, and what comes out is a single unbounded real number trained so that relevant pairs score higher than irrelevant ones. A number of that kind is a logit: it runs from minus infinity to plus infinity and becomes a probability only after something squashes it into the zero-to-one range. Depending on how the model was trained, that number is the input to a sigmoid over a binary relevance label — the sigmoid being the S-shaped function that does the squashing — or a margin in a pairwise ranking loss, or a temperature-scaled logit in a listwise softmax over one positive and several negatives. Only the first of those even attempts to make the absolute value mean something, and even there the meaning is calibrated to the training distribution rather than to yours.

It is worth seeing why the other two leave the level undetermined, because the argument is two lines of algebra and it is the reason the whole chapter exists. A pairwise ranking loss reads the difference between a positive pair's score and a negative pair's score. Add the same constant to both and the difference is unchanged, so the loss cannot tell that anything happened. A listwise softmax divides each exponentiated score by the sum of the exponentiated scores in the same list; adding a constant to every score multiplies each exponential by the same factor, that factor appears in the numerator and in the denominator, and it cancels. In both cases there is a whole family of score levels the objective scores identically, and gradient descent has no reason to prefer one member of that family over another. Whatever level the trained network settles on for a given query is therefore an artefact of the function it happened to learn, and not a quantity anybody specified.

Two consequences follow directly. Within one query's candidate list the ordering is trustworthy, because ordering is the only thing those losses ever constrained. Across two queries the levels are not comparable, because the constant that would have made them comparable was never pinned down.

{
  "query": "does the free plan support SAML",
  "reranked": [
    {"rank": 1, "score": -8.42, "doc": "SSO is available on Enterprise; see the SAML setup guide."},
    {"rank": 2, "score": -9.11, "doc": "The free plan includes up to three seats and community support."},
    {"rank": 3, "score": -9.87, "doc": "Free-plan workspaces can enable two-factor authentication."}
  ],
  "note": "same model, a query it can answer",
  "reranked_good_query": [
    {"rank": 1, "score": 9.63, "doc": "SAML SSO requires an Enterprise plan and an IdP metadata URL."},
    {"rank": 2, "score": 4.10, "doc": "SSO sessions expire after 12 hours by default."}
  ]
}

The two lists in that trace come from the same model on the same corpus, and the top scores differ by eighteen points. That figure is the arithmetic the previous paragraph predicted, made visible: the same network, the same weights, two queries, and the level of the whole score column has moved by more than the entire spread within either list. The ordering within each list is trustworthy — the first list really does have the least-bad passage at rank 1 — and it is the only thing the ranking loss was ever asked to get right.

Scales also differ wildly between models, which is why a threshold copied from a blog post is meaningless: ms-marco-MiniLM reranker scores in this corpus span roughly −12 to +11, bge-reranker-base emits sigmoid outputs in $[0, 1]$ that cluster hard near the ends, and hosted APIs typically return a normalised 0-to-1 relevance that is monotonic in the underlying logit but is not a probability of anything. None of the three can be compared to either of the others, and none survives a model version bump unchanged.

Calibrating against judged pairs#

Turning the score into something you can set a bar on takes the judged set and a binning exercise. Score every judged query-document pair with the deployed reranker, bucket by score, and count how often the human said relevant. What comes back is the empirical probability of relevance given the score, on your data, for this model version.

Six thousand eight hundred pairs below −8, and 27 of them relevant. That count comes straight out of the row: four in a thousand of six thousand eight hundred and forty-two pairs is twenty-seven pairs, which is the arithmetic that makes the row's verdict legible — a fraction that small is easier to disbelieve than a count that small. The second column is worth reading downwards as well as across. The bottom row holds more than twice as many pairs as the other five rows put together, so the typical query-document pair the reranker sees is not a near miss or a coin flip; it is a pair from the region where almost nothing is relevant.

That bottom row is where the SAML query's entire candidate list lived. The mapping is monotonic and reasonably smooth, so it fits well with either a one-parameter Platt scaling, which fits a single sigmoid to the scores, or an isotonic regression, which fits any curve at all provided it never decreases, and the fitted function is what converts a threshold conversation from "score above −2, I think" into "keep passages with at least a 40% chance of being relevant". Monotonicity and smoothness are measured properties of this table rather than guarantees the architecture provides, which is why the binning exercise comes before the curve fitting and not after it.

from sklearn.isotonic import IsotonicRegression
import numpy as np, json

def fit_calibration(judged_pairs, ce):
    """judged_pairs: [(query, passage, label_0_or_1), ...] from the golden set."""
    scores = ce.predict([(q, p) for q, p, _ in judged_pairs], batch_size=64)
    labels = np.array([y for _, _, y in judged_pairs], dtype=float)
    iso = IsotonicRegression(out_of_bounds="clip", y_min=0.0, y_max=1.0)
    iso.fit(scores, labels)
    return iso

def select(candidates, ce, iso, query, min_prob=0.40, n_max=10):
    raw = ce.predict([(query, c["text"]) for c in candidates], batch_size=64)
    probs = iso.predict(raw)
    keep = [(c, float(p)) for c, p in zip(candidates, probs) if p >= min_prob]
    keep.sort(key=lambda cp: -cp[1])
    return keep[:n_max]          # may return fewer than n_max, or nothing

The min_prob argument is where a product decision gets written down. At 0.40 the system says a passage may enter the prompt when a human would more likely than not have called it irrelevant, which is generous and appropriate when the generator is good at ignoring context it cannot use. At 0.7 it is strict, ships fewer passages, and abstains more often. The threshold is a dial on the precision-recall trade for the final list, and the calibration is what makes the dial's positions mean the same thing next quarter.

Why a relative cut-off is the wrong defence#

A common shortcut skips the judged pairs and thresholds relative to the top score instead — keep every candidate within, say, 20% of the best one, or every candidate above the mean plus one standard deviation of this query's scores. It requires no labels, which is its whole appeal, and it is worth being precise about why it fails, because the failure is a general shape rather than a detail of this system.

A relative rule reads only the spread of one query's own scores. The dangerous property it is supposed to catch is whether anything in the list is relevant at all, and that property lives in the level of the scores rather than in their spread. Compare the two lists in the trace above. One is led by 9.63 with a second at 4.10; the other is led by −8.42 with a second at −9.11. The gap between first and second is wide in one list and narrow in the other, and in neither case does the gap say which query has an answer in it. A rule computed from the spread therefore returns a confident-looking selection in both, and returns it for the same reason in both. The defence does not move when the thing it must discriminate moves.

The SAML query's scores were −8.4, −9.1 and −9.9, tightly clustered near the bottom, so a relative rule keeps all three and reports a confident-looking spread. Relative rules encode the assumption that at least one candidate is good and only the cut point is in question, and the queries that hurt you are the ones where that assumption is false.

Collecting the judged pairs is less work than it sounds, because the golden set already built for retrieval evaluation supplies most of them. Every query in it, run through the current pipeline at depth 50, produces 50 pairs of which a handful are already labelled; the rest can be sampled by score stratum rather than uniformly, since a random sample of 2,000 pairs from a depth-50 run is roughly 96% obvious negatives and teaches the calibration nothing about the region near the bar. Two hundred pairs drawn evenly across score deciles pin the curve better than two thousand drawn at random, and that is an afternoon of labelling rather than a project.

Sampling that way is legitimate for a specific reason worth stating, since deliberately unbalancing a sample usually biases whatever you compute from it. The quantity being estimated is the fraction relevant within a score band, and drawing more pairs from one band than another changes how many pairs land in each row of the table without changing that fraction inside any row. Stratifying on the score is safe precisely because the score is the thing being conditioned on. Stratifying on anything else — document length, say, or product area — would not be.

What changes when n stops being constant#

Applying a 0.40 bar to the same 200-query evaluation set redistributed the output sizes considerably. The median query returned 6 passages instead of 10, 27 queries returned fewer than 3, and 14 queries — the ones whose relevant document was never retrieved, the same population the ceiling argument in chapter 2 counted when 38% of queries had no relevant document anywhere in their candidate list — returned zero.

The first two rows are one measurement seen twice, and checking that they close is the cheapest validation available on a table like this. Dividing the fixed-n column's 2,200 context tokens by its 10 passages gives a mean passage of a little over two hundred tokens. Multiplying that same passage length by the calibrated column's 6.3 passages gives a shade under fourteen hundred tokens, which is the 1,390 in the cell beside it. No residual, so the token saving is entirely accounted for by passages that were dropped, and none of it by passages that got shorter. That matters for what happens next: a saving made of dropped passages can be undone by lowering the bar, whereas a saving made of shorter passages could not be.

The last row is the one the SAML ticket was about, and the drop comes almost entirely from queries that now return nothing and route to a different prompt — one that tells the model no supporting material was found and instructs it to say so. Sending zero passages is not a failure state to be engineered around; it is the accurate output for a query the corpus does not answer, and it is only reachable if something in the pipeline is permitted to say no.

Fourteen queries in two hundred is seven in every hundred, and it is worth noticing that this is smaller than the share of queries with no relevant document anywhere in their list. The two are not the same set. A query whose correct answer was never retrieved can still be handed a passage plausible enough to clear a 0.40 bar, and on those queries the bar changes nothing at all. What the bar catches is the subset where the candidate list is not merely wrong but visibly weak.

The 3.1% in the fourth row is the price. Nine relevant passages across 200 queries fell below the bar, mostly long passages whose relevant span sat past the truncation window and scored badly for that reason. Lowering the bar to 0.25 recovers seven of them and lets 40 more irrelevant passages through, and which of those trades is correct depends on whether the downstream consumer suffers more from missing evidence or from misleading evidence. A generator answering compliance questions and a search UI listing links have opposite answers.

Variable n also has integration costs that are worth naming before they surprise someone. Prompt sizes stop being predictable, so a token budget that was static becomes a function of the query. Evaluation gets fiddlier, since precision@10 over a list that is sometimes three long needs a stated convention. Caching keyed on a fixed result-set shape breaks. None of these is difficult, and all of them are easier to handle at design time than after the first oversized prompt gets rejected by the model's context limit.

An empty list has to survive the code that receives it#

Two weeks after the bar shipped, the ungrounded-answer rate on the no-relevant-document queries drifted from 17% back to 44%, with no configuration change, no model version bump and no corpus change. The retrieval service's own metrics were unmoved: 7.2% of queries were still returning zero passages, exactly as they had on the day the threshold went in. The regression was in the caller. A branch in the orchestration layer, written a year earlier, treated an empty context list as a retrieval fault, logged a warning, and retried the query with reranking disabled — which returned the original top 10 by embedding similarity, SAML passage at −8.4 included.

That production rate is close enough to the judged-set figure to be worth one sentence. Fourteen queries in two hundred is seven in a hundred, and the live meter reads 7.2%, so a curated set of two hundred queries and a week of real traffic agree about how often this pipeline has nothing to say. Two instruments landing on the same number is what licenses using the judged set to predict the abstention rate before shipping, which is a thing the judged set is not entitled to do for most metrics.

Nobody wrote that branch to defeat the bar. It was there because an empty list used to mean the search backend had timed out, which was true for the entire life of the system until the day a component was given permission to return nothing on purpose. Calibration changes what empty means without changing how empty is represented, and a list of length zero carries no account of why it is that length. Every defensive fallback downstream of the reranker was built against the old meaning, and each one quietly restores the behaviour the bar was installed to prevent.

The repair is to make abstention and failure different values rather than the same value:

@dataclass
class Selection:
    status: Literal["ok", "below_bar", "error"]
    passages: list[Passage]
    best_prob: float | None   # None only when status == "error"
    bar: float

A caller that pattern-matches on status cannot silently treat below_bar as something to retry, and the two extra fields answer the question an on-call engineer actually has when they see an empty result, which is whether nothing was retrieved or nothing cleared 0.40 and by how much. A below_bar with best_prob of 0.39 is a bar that could reasonably be argued with; one with best_prob of 0.004 is the corpus saying it does not contain the answer.

The below_bar rate then becomes a metric worth watching in its own right, because it moves for causes the quality dashboard cannot attribute. When a re-chunking pass halved median passage length, the rate went from 7.2% to 21% within an hour of the reindex finishing — calibration drift arriving as a step change on a graph rather than as a slow reputational decline in answer quality. An abstention rate is a leading indicator; the answer-quality metric that eventually notices is a lagging one, and the gap between them is measured in weeks of shipped answers.

Calibration drift, and what a bar cannot recover#

Calibration drifts, and it drifts for reasons that have nothing to do with the reranker. Re-chunking the corpus changes the length distribution of passages, which changes the score distribution, which invalidates the mapping. Adding a new document type does the same. Upgrading the reranker obviously does. The mapping is fitted on a snapshot and should be refitted whenever the model, the chunker, or the corpus composition changes, with the previous fit kept around so the shift can be inspected rather than merely absorbed — a threshold that silently became stricter looks exactly like a retrieval regression on a dashboard.

Nor does a bar recover anything. It cannot promote a document that was never retrieved, and on the fourteen queries that now return zero it has not found the answer; it has replaced a confident wrong answer with an honest empty one. Those are different product behaviours with different user costs, and the improvement in the last table's bottom row is a reduction in harm rather than an increase in coverage. The queries are still failing, and they are failing for first-stage reasons.

There is a tempting middle position that should be named before somebody adopts it, which is to keep the calibration and drop the abstention — to log the probability, show it in the interface, and still send ten passages. That buys the observability without the behaviour change, and it leaves the failure exactly where the SAML ticket found it, because a number nothing branches on is a number that cannot prevent anything. The value in the mapping is realised at the point where a piece of code refuses to return something.

What the score column supports, then, is narrower than it looks and more useful than nothing. Within one query it is a reliable ordering, because ordering is what the training objective constrained. Across queries it means whatever a binning exercise on your own judged pairs says it means, and that mapping has to be refitted when the model, the chunker or the corpus moves. Everything a bar can do rests on that second sentence, which is why a threshold set from a raw score somebody else published is a threshold set from nothing at all.

The threshold is also not a substitute for measuring whether the reranker belongs in the pipeline at all. Everything in this chapter improves the second stage's output given that a second stage exists, and none of it addresses the question the proposal in chapter 2 was really asking, which is whether the relevance the reranker buys is worth what it costs to run. Answering that means putting the quality numbers and the latency numbers into the same experiment, because measured apart they never meet.