A cosine score is calibrated to one model and one index

Two embedding models can have the same output width and still produce vectors that cannot be compared. This is the fact that makes model migrations dangerous in a way that other dependency upgrades are not, because the type system agrees that the new vectors fit where the old ones went. A vector(1536) column accepts a 1,536-component array whatever wrote it, an HNSW index computes a cosine between any two rows it holds, and neither of them has any way to ask which model a row came from.

The reason no such check exists is worth being exact about, since it is the whole chapter in one sentence. An embedding model's training objective, as chapter 2 set out, only ever compares vectors the model itself produced; nothing in the loss refers to any other model's output, and nothing constrains where a given meaning ends up. Two training runs therefore have no reason to agree on any direction, even with identical architecture and identical data. That makes a vector a position in one space, meaningful against the other positions in that space and undefined against anything else — and it makes every number derived from those positions, including a threshold somebody hard-coded once, a property of that space too.

The cosine between two models' vectors is not a small number, it is not a number#

same_text = "Refunds are issued to the original payment method within 5 business days."

v_a = embed(same_text, model="text-embedding-ada-002")      # 1536-d
v_b = embed(same_text, model="text-embedding-3-small")      # 1536-d

print(round(float(v_a @ v_b), 4))          # 0.0181

# what two unrelated directions in 1536-d score
rng = np.random.default_rng(0)
R = rng.normal(size=(10000, 1536)); R /= np.linalg.norm(R, axis=1, keepdims=True)
print(round(float(np.abs(R[::2] @ R[1::2].T).diagonal().mean()), 4))   # 0.0204

The same sentence, embedded by two models, scores 0.0181 against itself. To know whether that is small, it has to be compared against what pure chance produces, and the second half of the snippet measures exactly that by drawing ten thousand unit vectors at random and averaging the absolute cosine of five thousand disjoint pairs.

The measured 0.0204 can also be derived, and the agreement between the two routes is what makes it trustworthy. Randomly drawn unit vectors in 1,536 dimensions have a cosine distributed around zero with a standard deviation of 1/√1536 ≈ 0.0255 — the spread narrows as the dimension grows, which is the formal version of the observation that two random directions in a very high-dimensional space are almost always nearly perpendicular. The code averaged absolute values rather than signed ones, and folding a roughly-Gaussian distribution about zero leaves a mean of about four fifths of its standard deviation. Four fifths of 0.0255 is about 0.020, which is the figure the simulation printed. Theory and measurement land on the same number by different routes, so neither is a coincidence of the seed.

Set the observation against that bar. The observed 0.0181 is below the mean absolute cosine of two directions chosen at random, which is what the models produced from the point of view of each other. It is not that the two models disagree about this sentence. It is that neither one's output carries any information about the other's.

Chapter 3's rotation argument explains why. The loss depends only on relationships within one model's output, so every training run settles into its own arbitrary arrangement of directions, and two runs have no reason to agree on any of them even with identical architecture and data. Two models of matching dimension are two 1,536-dimensional spaces that share a shape and nothing else, and the fact that the dot product is computable is a property of Python rather than of the models. There is research on learning a map between embedding spaces after the fact, by fitting an orthogonal transform on paired texts, and it recovers a useful fraction of the alignment on some pairs of models; that is a fitted artefact requiring paired data and evaluation, and it is not something a database's distance operator, pgvector's <=> among them, does for you.

This is where the defence that a migration usually relies on gives way. The check that stands between a mixed index and production is the dimension check — performed by the column type, by the client library, and by whoever wrote "same 1,536 dimensions so no schema change" in a ticket. That check returns the same verdict whether the row was written by the old model, the new model, or a third model nobody has heard of, because all three emit 1,536 floats. A guard whose output does not vary with the thing it is supposed to discriminate is not a weak guard; it carries no information at all about the question being asked of it, and the confidence it produces is entirely borrowed from the fact that it ran.

Everything that broke, in the order it began hurting#

The ordering in that table is by time-to-symptom rather than by severity, and the two run in opposite directions. Rows 1 and 2 begin on the first query and are the ones users notice; rows 5 and 6 take weeks and are the ones that make the situation hard to repair, because by then the evidence needed to measure the damage has itself been produced under the wrong geometry.

The first row is the mechanism behind the visible symptom. With a small share of a corpus re-embedded — 4%, three weeks into a rolling backfill — the top ten for a typical query contains nine or ten of those new chunks, because everything else in the index is effectively noise relative to the query vector. That is not a competition the old documents lose narrowly. A query vector from model B scores around 0.02 against every A-era row, so the entire old population is bunched at the chance level while the new population spreads across the model's real range, and any new chunk with even a loose topical connection outranks the best old chunk in the corpus. Nothing errors, and a system that returns plausible recent documents for every query looks like it is working until someone asks it about last year.

The second row is the one that produces the more dramatic graph, and it comes from a property of the old model that is easy to miss. ada-002 has a famously compressed score range: unrelated text pairs land around 0.70, related pairs around 0.85, so a cutoff of 0.75 sat sensibly in the middle of that band. Under text-embedding-3-small the same unrelated pairs land near 0.05 and strongly related pairs around 0.55 to 0.75. A threshold of 0.75 that used to admit most reasonable matches now admits almost nothing, and the constant did not change while everything it referred to did.

Both columns describe the same pairs of sentences, which is what makes the table readable as gaps. Under ada-002 the whole usable span, from unrelated up to paraphrase, is two tenths of the scale. Under text-embedding-3-small the same span is over four fifths of it. The second model is not scoring these pairs "lower"; it is spending far more of the available range on them, which is the property that makes a cutoff possible at all.

The two cutoff values then stop looking arbitrary. Each sits inside its own model's gap between the unrelated level and the same-topic level, and each sits around three quarters of the way up that gap. That is the constant which actually travels between models: not the number, but its position within a distribution the model defines. Anyone who ports the number has ported a percentile from one distribution and applied it to another, and 0.75 in the second model's band is far past the paraphrase level rather than below the same-topic level. A cutoff is a point on a distribution, the distribution belongs to a model measured on a corpus, and moving either invalidates it — which the vector arithmetic topic argued for a single model and which compounds when the model changes underneath a stored constant.

Row three deserves attention because it survives the obvious fix. HNSW builds a navigable graph whose edges were chosen using distances between the vectors present at insert time, so a graph grown half under A's geometry and half under B's has neighbourhoods that make sense only within each half, and searches entering through an A-era entry point can fail to reach the B-era region at all. Re-embedding the old documents later updates the vectors while leaving the graph structure that was built from the old ones, so a full index rebuild is part of the repair rather than an optimisation.

Detecting the situation after the fact takes one query, once the model is recorded per row. Group the top ten results of a day's traffic by the model that produced each vector and compare that distribution against the model's share of the corpus: 97% of top-ten slots going to vectors from a model that wrote 4% of the rows is the failure, stated as a ratio. Under a healthy index those two percentages track each other loosely, because a model that wrote a twenty-fifth of the corpus should win roughly a twenty-fifth of the slots plus or minus whatever recency bias the content has. A ratio near twenty-four to one between them is not bias; it is one population being scored on a scale the query cannot reach. The same check catches a partially completed backfill, which produces the identical symptom at smaller scale and is otherwise indistinguishable from ordinary recency bias.

What the row should have carried#

The defect that makes all six possible is that a vector was stored without recording what produced it. Everything else follows from the system's inability to answer "which model wrote this row".

CREATE TABLE chunks (
  id            bigserial PRIMARY KEY,
  doc_id        bigint NOT NULL,
  text          text NOT NULL,
  embedding     vector(1536) NOT NULL,
  embed_model   text NOT NULL,      -- 'text-embedding-3-small'
  embed_dim     int  NOT NULL,      -- 1536
  normalised    bool NOT NULL,
  chunker_ver   text NOT NULL       -- chunking changes results as much as models do
);

CREATE INDEX chunks_hnsw_3small
  ON chunks USING hnsw (embedding vector_cosine_ops)
  WHERE embed_model = 'text-embedding-3-small';

The partial index is the load-bearing part, a partial index being one that covers only the rows matching its WHERE clause and is used only by queries carrying that same condition. It makes every search state the model it was built for, so a query embedded by B and run against A's index either uses the wrong index and returns obviously wrong results in staging, or — with the search function requiring the model name as a parameter — refuses. What that buys, in the terms of the section above, is a guard whose answer varies with the thing being asked: the model name discriminates where the dimension count cannot. A pipeline that also records the chunker version can answer why last month's numbers differ, since re-chunking moves recall by as much as a model swap does and is even easier to do without noticing.

Doing the swap so that it works#

The migration that produces no incident runs both models for a period and cuts over once, which costs storage and time rather than correctness.

The numbers make this tractable to plan, and each of them is one multiplication. Re-embedding 4 million chunks averaging 350 tokens is 1.4 billion tokens, since four million times three hundred and fifty is fourteen hundred million. At $0.02 per million that is fourteen hundred multiplied by two hundredths of a dollar, which comes to about $28. At a rate limit of 1 million tokens per minute it is fourteen hundred minutes, which is roughly 23 hours of wall clock — so the backfill is a day-long job rather than a quarter's project, and the cost is not what decides anything. Storage doubles for the duration of the dual-write window, from 24.6 GB of vectors to 49.2 GB, plus two graph indexes, and that peak is the constraint that decides whether the migration needs a bigger instance.

The shadow phase is what makes the cutover a decision rather than a hope. Running both indexes against the same live queries and logging the two result sets gives the overlap between them and, on the labelled query set, the recall of each. It also produces the pairs needed to recalibrate the threshold, since the constant has to be refitted on B before B serves anything — the cutoff in the table above moves from 0.78 to 0.32 for this pair of models, and no amount of reasoning about "similarity" would have suggested that direction.

The swap itself was fine#

None of this argues for staying on an older model. text-embedding-3-small is five times cheaper per token than ada-002, scores better on retrieval benchmarks, and supports dimension truncation the older model does not; the ticket's reasoning was sound and the execution left an index holding two geometries. What made it expensive was treating the vector as a value rather than as a value plus the model that defines it.

Two smaller cases are worth separating out, because they mark where the rule stops being absolute. Truncated vectors from a Matryoshka-trained model are comparable to each other provided both sides are cut to the same length and renormalised, since they come from one space; the rule is about spaces rather than about byte counts, and truncation stays inside a space. Mixing a 1,536-dimensional and a truncated 512-dimensional vector from that same model is still a category error, and the dimension mismatch will at least fail loudly — which is the one configuration in this chapter where the dimension check happens to carry information. And a provider quietly repointing a model alias to a new build produces the same incompatibility with no ticket at all, which is why the schema above records a revision rather than a friendly name.

Aggregate comparisons across models are equally void. A dashboard showing mean top-1 similarity rising from 0.81 to 0.86 after a model change reports that the second model's score distribution sits higher, and says nothing about whether users found what they wanted. A model can raise that mean by compressing its range upward, which the ada-002 row of the table above is an example of, and compression is the opposite of what a filter needs. The metrics that survive a model swap are the ones defined over outcomes — recall against labelled judgements, click-through, the polarity-correct rate from chapter 4 — because they are computed from documents and human judgements rather than from coordinates.

That is the whole of what a vector is: a fixed-length point whose position was arranged by a contrastive objective, whose axes carry no individual meaning, whose distances encode subject matter and not logical relation, and whose scores are interpretable only against the model and index that produced them. Each of those four clauses came from one of the chapters here, and together they say why a number pulled out of a vector database is not yet a fact: it is a reading from an instrument, and the instrument has to be named for the reading to mean anything.

What comes next is building on it — deciding chunk boundaries, combining dense retrieval with lexical search, choosing k, and measuring the result — with the ability to predict which of those decisions the geometry will reward and which it will quietly ignore.