Two embedding generations cannot share one namespace, even at equal dimensionality

An embedding generation is every vector in a corpus that was produced by one particular encoder — one model, at one pinned version, with one set of encoding conventions. An embedding model is a function from a piece of text to a fixed-length list of numbers, and what each position in that list means is decided entirely by training: the axes are learned rather than given, and they have no referent outside the model that produced them. A namespace is the unit inside a vector store within which distances are computed — one vector field on a stored point, carrying one dimensionality, one metric and one index. Anything sitting in a namespace is treated as mutually comparable with everything else there, and that treatment is an assumption the store makes rather than a property it verifies.

Upgrading an encoder is the most ordinary maintenance task in this area, and it is the one that fails without producing an error to look at. What moves instead is nDCG@10 — normalised discounted cumulative gain at ten, a ranking score between 0 and 1 that rewards putting the relevant chunks near the top of the ten results returned — which falls in proportion to how much of a corpus has been re-embedded, while error rates, latency and ingest throughput all stay where they were. Rebuilding an index over vectors that do not change is a capacity and recall problem (chapter 3, where tombstoned vectors keep spending search budget until compaction runs). Changing the vectors themselves is a different operation, and what it does to everything already stored shows up on no counter the store exposes.

Distance across generations is arithmetic without meaning#

An embedding model defines a space by training. The axes have no external referent, the geometry is a consequence of the objective, the data and the initialisation, and two models trained independently have no reason to agree about where anything goes. Cosine similarity between model A's vector for a sentence and model B's vector for the same sentence is not a measure of anything; it is the inner product of two vectors drawn from unrelated distributions.

That claim is worth deriving rather than asserting, because the number the comparison produces is not obviously meaningless — it is a real number in the range scores normally occupy, and it is stable across runs. Cosine similarity is the dot product of two vectors divided by both of their lengths, and when the vectors have already been scaled to length one, which is what normalize=True does in the listing below, the division is by one and the cosine is exactly the sum of the products of matched components. The first component of one vector multiplied by the first component of the other, the second by the second, and so on down all 1024 positions, with the products added up.

Within one model that sum means something, because both vectors were laid out against the same learned axes, and agreement position by position is what the training objective rewards. Across two models nothing lines the axes up. The seventeenth position of one model's output and the seventeenth position of the other's are unrelated quantities that happen to have been written at the same offset, so each product is as likely to come out positive as negative, and no mechanism pushes them one way. The signs cancel, and the total has no reason to be large in either direction.

Cancellation is a tendency rather than a guarantee, which is why the result is described by a spread and not by a single value, and the size of that spread follows from two steps. First, the size of a typical term: the components of a unit vector have squares that add to one, and that one is shared out over 1024 positions, so a typical square is one part in 1024 and a typical component is about one over the square root of 1024 in size. Multiplying two such components gives a product of about one part in 1024, carrying a random sign.

Second, what happens when 1024 such terms are added. If they all carried the same sign the total would be the count times one term, and it does not, because each new term is as likely to undo the running total as to extend it. A sum of independent contributions of random sign wanders rather than marches, and a wander of that kind covers a distance proportional to the square root of the number of steps rather than to the number of steps itself. So the total is about the square root of 1024, times the one-part-in-1024 size of a single term, which is one over the square root of 1024 — the quantity written 1/sqrt(d), where d is the dimensionality.

Putting the dimension in gives the figure. The square root of 1024 is thirty-two, so the spread is about one part in thirty-two, which written as a decimal is 0.031. The centre of that spread is zero, so a cross-generation score is not a small positive measurement of weak similarity; it is a draw from a distribution centred on nothing, with a standard deviation of 0.031.

texts = ["The refund window closes 30 days after delivery."] * 1
a = model_old.encode(texts, normalize=True)   # 1024-d
b = model_new.encode(texts, normalize=True)   # 1024-d

float(a[0] @ b[0])          # 0.017  — same sentence, two models
float(a[0] @ a_para[0])     # 0.883  — paraphrase, same model
float(a[0] @ a_rand[0])     # 0.041  — unrelated sentence, same model

Those three numbers are one comparison rather than three facts. The middle one is what makes a vector index work at all: 0.883 for a paraphrase scored inside a single model is the property the whole retrieval system rests on, that semantically close texts land close. The finding is the relation between the other two. The same sentence encoded by two models scores 0.017, and two unrelated sentences encoded by one model score 0.041, and both of those sit inside the spread that pure noise produces at this dimension, since 0.031 is one standard deviation of it. The cross-generation score is therefore not merely small. It is indistinguishable from the score of two sentences that have nothing to do with each other.

That is what "arithmetic without meaning" means concretely: the number exists, it is computable, it is reproducible, and it carries no information about the texts that produced it. The honest form of the claim is about indistinguishability rather than about one number being smaller than another — a single pair either way is one draw, and separating two distributions this close would take many pairs. That inability to separate them is the finding rather than a weakness of the measurement, because nothing downstream can use a quantity whose value would be equally likely if the two texts were identical or unrelated.

The partition that follows is exact, and it is worth adding up. 12% of a 6-million-chunk corpus is 720,000 chunks. The other 88% is 5.3 million. Those two add back to six million with nothing left over, because every chunk was written by one encoder or the other and no chunk was written by both. When a query is embedded with the new model, the 5.3 million old-generation vectors all score around 0.02 against it and are effectively invisible, so the search is a search over 720,000 chunks dressed up as a search over six million. Nothing in the response says so. The engine returns ten results with plausible scores, because ten results is what it was asked for and the top ten of a small corpus still exist.

The rule behind that arithmetic explains the shape of the damage over time. The fraction of a corpus a query cannot see is the complement of the share written by the generation its encoder belongs to. Queries encoded with the new model cannot see the not-yet-migrated remainder, which shrinks as the backfill proceeds; queries still encoded with the old model cannot see the migrated share, which grows. Either way the eval score moves in step with the migrated fraction, and it is the shape of that curve rather than its level that names the cause, because an ordinary relevance regression does not track a deployment's progress figure. One condition attaches: this holds while relevant chunks are spread across the two generations roughly in proportion to their sizes. Where re-embedding is triggered by edits, the migrated share is exactly the recently changed material, which is also what recent queries ask about, and the loss arrives steeper than the fraction suggests and shows up on new content first.

The reverse case is worse than either, and it shows up during a partial rollout of the query encoder — which is why splitting query traffic between the two encoders is not a mitigation. If some queries are embedded with the old model and some with the new, each query is in one of the two cases above and neither is safe. Two users asking the same question in the same minute get answers drawn from disjoint halves of the corpus, and the difference between the two answers is attributable to which pod served them rather than to anything about the question.

Two humps where a healthy index has one#

The cheapest way to see this in a live system is to plot the score distribution of returned candidates. A healthy single-generation index produces a unimodal spread of top-k scores — one hump, with a tail whose length depends on the query. A mixed index produces two humps: one in the normal range, and one clustered near zero with a width close to 1/sqrt(d). The derivation above says what that second width is, which makes the second hump identifiable rather than merely odd. It is not a population of weak matches, it is the noise floor drawn out at full size.

Averaging is what hides it, and the reason is a property of bimodal distributions rather than anything specific to embeddings. The mean of a distribution with two humps sits between them, in the gap where none of the observations actually are, and it moves gently as weight shifts from one hump to the other. So a mean score can drift by a few hundredths across a migration that has silenced most of a corpus, and it will never show the one feature that gives the diagnosis away, which is that there are two populations. The general lesson is that the shape of a score distribution is a diagnostic instrument, and any summary statistic that collapses it to one number destroys exactly the structure worth looking at.

Hybrid retrieval makes the symptom milder and the diagnosis harder. Reciprocal rank fusion combines a lexical ranking — BM25, which scores documents by weighted term overlap with the query — with a vector ranking, and it combines them by position rather than by score. A keyword match on an old-generation document therefore still surfaces it, whatever its vector score was, and the answer often looks acceptable. What has actually happened is that the vector half of the hybrid contributes nothing for 88% of the corpus, so the system has degraded to BM25 with a semantic garnish for most documents and full hybrid for the migrated remainder. Aggregate relevance metrics move by a few points, which reads as noise, and the failure is only visible if you evaluate the retrievers separately.

The failure is silent in exactly the cases you will hit#

The belief that lets this run for weeks is that the store validates what it is given, and the evidence for it is the dimensionality check. That check is a defence that is constant when the thing it must discriminate varies: it compares the width of an incoming vector against the width the collection was created with, and both generations produce 1024 numbers. It therefore reports "accepted" for a correctly encoded vector and for a vector drawn from an unrelated space, identically. A guard whose output does not change when the condition changes is not detecting the condition.

The mistake is tempting for a specific reason. Dimensionality mismatch is the one member of this family that fails loudly — push 1024 numbers at a collection built for 1536 and the write is rejected on the spot — and an engineer who has met that error has direct evidence that the store inspects vectors before accepting them. The generalisation from there is that the store validates vectors. What it validates is a shape.

The last three rows are the ones that catch teams who believe they are not doing a model migration at all. Matryoshka training arranges a model's output so that a prefix of it is itself a usable embedding at lower dimensionality, and the saving is real, but a vector truncated to a shorter prefix is comparable only with vectors truncated at the same place — two lengths of the same model's output are two spaces, exactly as two models are. An instruction-tuned retrieval model that expects query: on questions and passage: on documents produces measurably different vectors when the prefix is omitted, and a refactor that moves encoding into a shared helper is a plausible way to lose one of them. Normalisation mismatch is invisible under cosine, because cosine normalises anyway, and distorts dot-product rankings in favour of whichever batch has larger magnitudes. None of these three changes the model name in your config, so none of them trips a check that watches the model name.

What follows is that "which model produced this vector" has to be a property of the index rather than a fact about deployment history. In practice that means the collection carries a manifest, and the query path refuses to run against a collection whose manifest does not match the encoder it is holding.

@dataclass(frozen=True)
class EmbeddingFingerprint:
    model: str              # "voyage-3-large"
    revision: str           # provider version pin, not "latest"
    dim: int                # 1024
    normalized: bool        # True
    query_prefix: str       # "query: "
    doc_prefix: str         # "passage: "
    pooling: str            # "mean" | "cls" | "last-token"

    def key(self) -> str:
        return hashlib.sha256(json.dumps(asdict(self), sort_keys=True).encode()).hexdigest()[:12]

def search(collection: str, question: str, k: int = 10):
    manifest = store.get_manifest(collection)          # written at index creation
    if manifest.fingerprint_key != ENCODER.fingerprint.key():
        raise EncoderMismatch(f"{collection} was built by {manifest.fingerprint_key}")
    vec = ENCODER.encode(manifest.query_prefix + question)
    return store.search(collection, vec, limit=k)

Mechanically, the fingerprint hashes every field that changes the geometry — model, pinned revision, dimension, normalisation, both prefixes, pooling — which is why it covers more than the model name, and why revision is a version pin rather than latest. The second property is the one that does the work: it is stored with the index rather than derived from the running configuration. The consequences chain. A collection named chunks_v3_voyage3large_1024 has a manifest that hashes to 8f2c41ad91e0, written at index creation → the write path checks that hash against the encoder it is holding, using the same comparison the read path uses → so a service that thinks it is producing something else cannot write into that collection at all → so a three-week silent migration becomes a deployment that fails immediately and loudly on its first write. The cost is honest and small: one comparison per request against a cached manifest.

The word doing the work in all of this is namespace rather than collection. Several engines let a single point carry multiple named vectors, each with its own dimensionality, metric and index — Qdrant calls them named vectors, and the equivalent exists elsewhere as multiple vector fields on one document. Storing emb_v2 and emb_v3 on the same point is entirely sound, because a search names the field it searches and never computes a distance across the two. The property that separates the sound arrangement from the unsound one is whether a single distance computation can ever straddle the two spaces: on named vectors it cannot, by construction, and in one shared field it can and will. That pattern keeps payloads, filters and ids in one place, which removes a class of synchronisation work, and its honest cost is the storage of both generations simultaneously. What is unsound is one field holding vectors from two encoders, which is what a config change to an existing pipeline produces by default.

Adapters, and the limit of what they buy#

There is one legitimate way to compare across generations, and it is worth knowing precisely because it is often misapplied. If you have both models, you can embed the same set of texts with each and learn a map from the old space into the new one — a linear projection fitted on a few hundred thousand paired vectors, or a small MLP where the relationship is not linear enough. Applied to the old vectors, the map produces something that lives approximately in the new model's space, so a new-model query can search old content without re-embedding it. Drift-Adapter: Near Zero-Downtime Embedding Model Upgrades in Vector Databases measures this trade directly, and the honest summary is that adapters recover most but not all of the quality of true re-embedding, at a small fraction of the compute, which makes them a good bridge and a poor destination.

The reason they are a bridge is that the map is fitted, so it has error, and the error is not uniform: it is largest exactly where the two models disagree most, which is where the new model was supposed to be better. The adapter reproduces the new space best in the regions the two models already agreed about, and those are the regions that did not need upgrading. An adapter over a 6-million-chunk corpus can be trained in under an hour and applied in a single pass, letting you cut the query encoder over immediately and run the real backfill behind it, and the eventual state is still a fully re-embedded index. Teams that stop at the adapter have bought a permanent quality ceiling in exchange for a weekend.

The approach that does not work at all, and gets proposed in most migration discussions, is score calibration — normalising the score distributions of the two generations so they can be merged. Calibration can align the scale of two distributions, and that is the whole of what it can do. It cannot create a correspondence between two sets of coordinates that were never related to begin with. An old-generation vector scoring 0.02 against a new-generation query is not under-scaled; it is uninformative, and rescaling noise produces louder noise.

Re-embedding is not an update to an index. It produces a different index over the same documents, with its own fingerprint, its own graph, its own tuning and its own capacity requirement, and the old one keeps serving traffic the whole time. That framing settles several arguments before they start: there is no partial state to reason about, no window in which mixed generations coexist behind a single alias, and no ambiguity about what a query is searching, because a query resolves to exactly one collection whose manifest it has verified.

It also means the migration is bounded by a number nobody has computed yet. Building a second index over six million chunks — or, in the case the next chapter takes up, eighty million chunks against a provider's rate limit — requires embedding every chunk again through a provider that charges per token and enforces that limit, storing both copies while the build runs, and finishing before the corpus you snapshotted has changed out from under you. The bill is the easy part of that estimate and the least likely to be what stops you.