Unit length collapses three metrics into one ranking

Normalising a vector means dividing it component by component by its own length, so that the result has a norm of exactly 1 and points in the same direction as the original. Whether the vectors in a search index have had that done to them is the single condition that decides whether the choice of similarity metric matters at all. Under it, the three quantities from the previous chapter — a dot product and two lengths, combined three ways — stop being three independent things and become one thing seen from three angles. Without it, they are three different questions and can give three different answers about the same query.

This is worth settling early in a project because the metric is usually chosen once, by whoever created the index, and often by leaving a field at its default. If the vectors are unit length, that choice is genuinely free and nothing downstream can be affected by it. If they are not, the choice decides which documents come back, and it does so through a quantity — the length of each stored vector — that nobody looked at and that correlates with how long the underlying document was rather than with what it says.

What the arithmetic does when every norm is 1#

Set ‖a‖ = ‖b‖ = 1 and put those values into the definitions worked out in the last chapter. Cosine's denominator, ‖a‖ × ‖b‖, becomes 1 × 1 = 1, so the cosine of the pair is just the dot product:

cos(a, b) = (a · b) / (1 × 1) = a · b

Not merely proportional — numerically identical, to the last decimal place your floats carry. Any ordering produced by one is the same ordering produced by the other, because they are the same list of numbers.

The Euclidean identity collapses just as far. Substituting into ‖a − b‖² = ‖a‖² + ‖b‖² − 2(a · b):

‖a − b‖² = 1 + 1 − 2(a · b) = 2 − 2 cos(a, b)

Both squared-norm terms became 1 for the same reason the cosine denominator did, and the dot product term stayed, so the whole expression now has exactly one moving part. Squared distance is an affine function of cosine with a negative slope, which means it is strictly decreasing in cosine over the entire range. When cosine goes up, squared distance goes down; when squared distance goes up, cosine goes down. Since the square root is itself increasing, the same holds for plain Euclidean distance — taking the root of two numbers cannot reverse which of them is larger. Sorting a candidate list by ascending L2 distance and sorting it by descending cosine produce the identical sequence, and neither one can put a document in a different place from the other.

Two worked values make the mapping concrete rather than abstract. Two vectors at cosine 0.90 sit exactly sqrt(2 − 1.8) = 0.447 apart, since twice 0.90 is 1.8 and two less 1.8 is two tenths, whose square root is 0.447. Two at cosine 0.50 sit sqrt(1.0) = 1.0 apart by the same substitution. Notice that the mapping is not linear: cosine fell by four tenths and the distance rather more than doubled, because a square root stretches small numbers relative to large ones. That is the reason to say "strictly decreasing" rather than "proportional". Ranking is preserved exactly; the spacing between adjacent scores is not, and anything reading gaps between scores rather than their order will read different gaps under the two metrics.

This is the answer to a question that comes up in interviews and in production postmortems in the same words: cosine and dot product return the same ranking exactly when the vectors are unit length, because that is the condition under which the norms cosine divides by are 1. If a system's vectors are all normalised, flipping the index from cosine to Euclidean is a no-op for ranking, and the only thing that changes is what number gets printed next to each result.

Three metrics, three different answers#

Drop the condition and watch it come apart. Take a normalised query vector q and four documents from a support corpus, with their norms and their dot products against q as the raw material:

Every column comes from the same three ingredients, and only the two middle ones were measured. The cosine column is q · d divided by ‖d‖, since q is already unit length; the L2 column is sqrt(1 + ‖d‖² − 2(q · d)), which is the general identity with ‖q‖² replaced by 1 because the query contributes a factor of one wherever its norm appears. Row A checks in a line: 3.20 over 4.1 is 0.780, and one plus 4.1 squared less twice 3.20 leaves a shade over eleven and a third, whose square root is 3.378. Every other row reconciles the same way with no residual, which is the point worth taking from the table — the last two columns contain no information the first two did not already have. And the three columns disagree about what comes first:

Dot product ranks the corpus almost exactly backwards from cosine, and Euclidean produces a third order that matches neither. The mechanism is visible in the norms, and it is worth stating as a comparison of spreads rather than as an impression. The dot product is the norm multiplied by the cosine, so which of the two factors decides the ranking is settled by which one varies more across the corpus. Down the cosine column the best and worst differ by a ratio of about four to three. Down the norm column they differ by a factor of three. The norm's variation is several times larger, so it wins, and the ordering it produces is an ordering by length with a little relevance mixed in.

That comparison also says when dot product would be harmless: on a corpus whose document norms varied less than its cosines did, the reordering would be small. Chunked corpora with uniform passage lengths come close to that condition, which is precisely why the defect hides in a test set of similar chunks and appears in production.

Euclidean distance penalises the same magnitude from the other side, pushing long documents away simply for being far from the origin, which is why B — genuinely more relevant than C — lands at rank 3. The ‖d‖² term in the identity grows as the square of the norm while the q · d term grows only in proportion to it, so for a long document the squared-norm term dominates and distance is close to being a measure of document length alone.

The connection between document length and norm is worth being precise about, because it is the mechanism that makes the reordering systematic rather than random. Most sentence encoders produce a document vector by mean-pooling the token vectors from the final layer. If the token vectors were independent and identically distributed with mean zero, averaging more of them would shrink the norm rather than grow it, so the observed growth comes from the fact that they are not independent: they share a large common component, the anisotropic direction every input projects onto. Pooling more tokens estimates that shared component more stably and adds more distinct content directions on top of it, and models trained without any normalisation in the loss have nothing pushing back on the resulting magnitude. The 4.1 and 12.4 in the table are drawn from a real spread; on a corpus of 200-to-3,000-word documents through a -dot- style model, norms commonly range over a factor of three, which is more than enough to dominate cosine differences of 0.1 or 0.2.

Now normalise every document vector, dividing each by its own norm, and score the same query again:

One ranking: A, B, C, D, under all three settings. The cosine column is unchanged from the first table, which is the definition of cosine doing its job — it had already divided the norms out, so removing them changed nothing. The L2 column is new, and every entry in it is sqrt(2 − 2 cos) applied to the column beside it: two less twice 0.780 is a little over four tenths, and the square root of that is 0.663. The L2 numbers are different numbers from the cosine ones, and they are not on the same scale, but they are a strictly decreasing function of them, so they sort identically. Three index configurations, one result set.

Running that comparison on your own corpus takes about twenty lines:

import numpy as np

# Q: (n_queries, dim), D: (n_docs, dim) — raw model output, not normalised
def topk(scores, k=10, ascending=False):
    order = np.argsort(scores, axis=1)
    return order[:, :k] if ascending else order[:, ::-1][:, :k]

def unit(X):
    return X / np.linalg.norm(X, axis=1, keepdims=True)

raw_dot    = topk(Q @ D.T)
raw_cos    = topk(unit(Q) @ unit(D).T)
norm_dot   = topk(unit(Q) @ unit(D).T)          # identical to raw_cos
norm_l2    = topk(
    np.linalg.norm(unit(Q)[:, None, :] - unit(D)[None, :, :], axis=2),
    ascending=True,
)

def rank1_agreement(x, y):
    return float(np.mean(x[:, 0] == y[:, 0]))

print("dot vs cosine, unnormalised:", rank1_agreement(raw_dot, raw_cos))
print("dot vs cosine, normalised:  ", rank1_agreement(norm_dot, raw_cos))
print("cosine vs L2, normalised:   ", rank1_agreement(norm_l2, raw_cos))

# dot vs cosine, unnormalised: 0.41
# dot vs cosine, normalised:   1.0
# cosine vs L2, normalised:    1.0

The two 1.0 values are guaranteed by the algebra and are worth asserting in a test rather than merely printing, because a value below 1.0 there means the normalisation did not actually happen everywhere you believed it did. That is the useful property of an assertion on a derived identity: it cannot fail for a reason to do with the data, so a failure localises the bug to the code path immediately. The 0.41 is the interesting one: on a corpus with a wide spread of document lengths, changing the metric without normalising moved the top result for 59 percent of queries, which is one hundred percent less the 0.41 that agreed. On a corpus of uniformly sized chunks the same measurement typically lands between 0.85 and 0.95, which is precisely why this defect survives a small test set and shows up in production.

Which side of the comparison has to be normalised#

The tables above normalised the query and left the documents alone, and that asymmetry is not incidental. Within a single query, the query's own norm is a constant applied to every candidate score — q · d = ‖q‖ ‖d‖ cos(q, d), and ‖q‖ does not vary as d ranges over the corpus. Multiplying every score in a list by the same positive constant cannot reorder the list, so normalising the query alone changes nothing about which document comes first. All of the reordering in this chapter comes from the document norms, because those vary from row to row. Dividing a list by a constant is a rescaling; dividing it by a quantity that differs per element is a reordering, and the two operations look identical in code.

That has a practical consequence and a trap in it. The practical consequence is that if you are only ever going to look at ranks, the query-side normalisation is optional and the document-side normalisation is the one that decides your results, which means a pipeline that carefully normalises queries and forgets documents has done the useless half of the job. Worse, it has done the half that produces reassurance: a code review sees unit(query_vector) in the retrieval path and concludes normalisation is handled. The trap is that ranks are not the only thing systems look at. The moment a score is compared against a fixed number — a relevance floor, an alerting threshold, a rule that says fall back to keyword search when nothing scores above some value — the scores from different queries are being placed on a common axis, and an unnormalised query multiplies that query's entire score list by an arbitrary factor. One query's 0.9 and another's 0.3 then differ partly because of how long the question was.

Getting normalisation right is also worth an assertion rather than a hope, because it is easy to do approximately. Dividing a float32 vector by its computed norm leaves a residual of about 1e-7, so a tolerance of 1e-6 is the right check and an exact equality test against 1.0 will fail on correct code. A tolerance of 1e-2, which people reach for when the strict check fails, is loose enough to pass vectors that were normalised under a different definition — L1, which divides by the sum of the absolute components rather than by the square root of the sum of their squares, instead of L2, or normalised before a subsequent projection layer rather than after it. A check that admits vectors under a definition the index does not use is a check whose answer does not depend on whether the property holds, which is the same as no check with a green tick attached. The tolerance itself is a decision worth making deliberately.

Choosing between them when they disagree#

The shallow version of this argument is that cosine and dot product differ only by a scale factor and therefore never matter in practice, so you should take whatever default the database offers. The scale factor is the problem. It is not one constant applied to all scores; it is a different divisor per document, equal to that document's own norm, and dividing a list of numbers by a varying quantity is exactly the operation that reorders it.

When they disagree, the question is whether magnitude carries information you want. For a corpus of similarly sized retrieval chunks embedded by a model trained with a cosine objective, magnitude carries noise, and cosine is what you want — the table above shows dot product elevating a document whose only advantage is size. There are models where the opposite holds, trained so that norm encodes something real such as how much answer-bearing content a passage carries, and for those the magnitude is signal that cosine deliberately throws away.

Neither case is decided by reasoning about it. It is decided by a labelled query set — a few hundred queries with a known-correct document for each — scored both ways, comparing recall@10 — the fraction of queries whose known-correct document appears anywhere in the top ten results — or nDCG@10, which is the same measurement weighted so that a correct document at rank 1 counts for more than one at rank 9. On the corpus above, cosine measured 0.81 recall@10 against dot product's 0.63, and the gap was concentrated entirely in queries whose answer lived in a short chunk. That concentration is the signature to look for, and it is more informative than the aggregate: a metric defect discriminates by document length, so slicing the evaluation by the length of the gold passage turns one number into a curve that names its own cause. An aggregate that moved for some other reason would not sort itself by length like that. The measurement takes an afternoon and it settles the question for that model and that corpus, which is the only scope in which it can be settled.

One thing normalisation does not do is make the choice disappear. Normalising every vector guarantees the three metrics agree with each other, and it does so by permanently discarding magnitude — if the model put something there, unit-normalising the corpus is the act of deleting it, and no index setting downstream can recover it. That is the honest statement of the trade: unit length buys you freedom from the metric question at the price of an irreversible decision about the norm, and the price is zero for models that put nothing in the norm and total for models that did.

So the real question is not which metric ranks better on a spot check but which one the model was built to be scored with, and that is a fact about the model rather than about your corpus. It is written down, in the one place most teams never look.