The migration bill is the window where both indexes are live

The deprecation notice gave nine months. The model behind a 400 million chunk index would stop accepting requests on 1 June, and the successor was better on every axis the announcement mentioned: 30% cheaper per million tokens, 1024 dimensions instead of 1536, and a clear win on the public retrieval benchmarks. The VP's reply in the thread was reasonable and completely wrong about the shape of the work — cheaper per token and smaller per vector, so this should pay for itself; can we do it in Q1 and book the saving.

Priced properly, the migration costs about $150,000 and returns about $3,200 a month, which is a payback period of just under four years. It is still the right thing to do, because the alternative is an index served by an API that returns 410 Gone in nine months. The reason to have the numbers is that a migration sold as a saving gets funded like a saving, which means a small team, no evaluation budget, and a schedule with no room in it.

Itemising a 400 million chunk migration#

The corpus is 400 million chunks averaging 410 billed tokens, so re-embedding is 164 billion tokens. At the successor's $0.07 per million that is $11,480, and it is the line everybody computes first because it is the only one with a published unit price attached.

RF in those rows is the replication factor from chapter 4, so the new index carries a single copy while it is being built and validated and is brought to three only when it has to serve traffic. The re-embedding is 7.6% of the project. The storage and serving lines together are $40,000, and they are the ones most often absent from the plan entirely, because they are not a purchase — nobody signs an invoice for "two indexes at once", it just shows up as a larger cluster for a quarter and gets attributed to growth. The four rows in the middle of that table are the transition, and the transition is where the money goes.

Chapter 4 worked out why the peak lands near 1.73 times the current footprint rather than the 0.67 that the dimension reduction suggests. What the table adds is that the peak is not a single instant but a period with a chosen length, and that its cost is roughly linear in that length. Eight weeks of dual residency at reduced replication plus four weeks at full replication is a schedule someone decided on, and shortening it is a lever with a price tag.

The transition has a shape, and every phase is a cost#

Diagram

The backfill bar is five weeks and that is not slack. At the effective 3.8 million tokens per minute a shared quota leaves for a batch job, 164 billion tokens is 719 hours, which is 30 days of continuous running before any interruption, requota, or corpus churn. Compressing it means buying quota, running a self-hosted fleet alongside, or sharding the work across projects, all of which are available and none of which are free. Meanwhile the corpus does not hold still for thirty days: at half a percent daily churn, two million chunks a day change underneath the job, so roughly 60 million chunk-writes of catch-up accumulate and have to be drained before the new index can be called complete.

The dual-serving phase is where the second query tier appears. Validating a new index against production traffic means running both retrieval paths for the same queries — the query text embedded by both models, two ANN searches, the results compared offline — with the new path shadowing the live one, answering real production queries whose results are recorded and never shown to anybody. That is a second serving fleet for as long as the comparison runs. Query-side embedding is cheap at these volumes: 40 million queries a month at 30 tokens each is 1.2 billion tokens, $84. The compute for a duplicated search tier is $6,048 a month, and it exists solely to produce evidence.

The rollback hold at the end is the phase people cut first and regret. Deleting the old index the day the new one takes traffic converts any post-cutover quality regression from a routing change into a four-week rebuild, and quality regressions are exactly the class of problem that surfaces a week after cutover, when the long tail of query types has had time to appear. Four weeks of retention is $8,990 and it buys an instant revert.

Because the transition lines are linear in time, the total is unusually sensitive to schedule slip in a way that a one-off purchase is not. Every extra week of dual residency at full replication is about $3,650, and every extra week of the second query tier is $1,510, so a migration that runs six weeks late costs roughly $31,000 more without anyone doing anything differently. That sensitivity is an argument for sequencing the phases so the expensive ones are short and the cheap ones absorb the uncertainty: the evaluation phase costs almost nothing per week and can be allowed to run long, while the full-replication window should be entered only when the cutover date is firm.

The lead times run the other way from the costs, which is why the plan has to name them early even though the spending is late. A quota increase large enough to compress a thirty-day backfill is a conversation with the provider that takes weeks; reserved capacity for a quarter-long peak has its own procurement path; and 675 hours of domain-expert judging — a person reading a query alongside a retrieved document and rating how well it answers, which is where a relevance judgement comes from — has to be scheduled around people who have other jobs. None of those need money committed in month one, and all of them need someone to have started asking.

Why cheaper per token is not a saving#

The steady state after the migration genuinely is cheaper. Storage falls from $8,986 to $6,528 a month on the same corpus, which is $2,458. Ongoing incremental embedding — half a percent daily churn on 400 million chunks is 820 million tokens a day, 24.6 billion a month — falls from $2,460 to $1,722, which is $738. Together, $3,196 a month.

Against $150,600 of transition, that is 47 months of payback, and the model behind it will almost certainly be deprecated again before then. So the answer to "why is this not a saving" is arithmetic rather than argument: the cheaper unit price applies to the ongoing trickle of new and edited documents, which is a small fraction of a percent of the corpus per day, while the transition cost applies to all 400 million chunks at once and to a quarter of doubled infrastructure. A 30% discount on the small recurring number does not fund a one-off equal to four years of it.

The correct frame for a deprecation-driven migration is that the cost is not optional and the question is how to spend it well. Where the frame matters most is in what gets cut. A project sold as a saving has no natural budget line for 27,000 relevance judgements, because judgements do not make anything cheaper. A project sold as a forced migration with a quality risk attached has an obvious reason to fund them.

What is different about a model change#

Every earlier chapter's machinery still applies — token-budgeted batching, a controlled share of quota, deterministic ids so the thirty-day backfill can survive a restart — with one change to the id derivation. The new vectors are a different population from the old ones, so they need their own id space or their own collection; writing them under the same ids as the old vectors would overwrite the index you are still serving from, and writing them into the same collection at a different width is not possible in most stores and is a correctness error in the ones where it is.

What genuinely does not carry over is comparison. A chunker change keeps the model fixed, so the old and new indexes can be evaluated on identical queries with exactly one variable moving, and a suspicious result has one candidate explanation. A model change moves the document encoder, the query encoder, the vector width, and often the appropriate distance metric and the appropriate ef_search — the number of candidates a single query is allowed to visit before the search stops — for a given recall target, all at once. Tuning the new index badly and tuning it well can differ by more than the model difference itself, which means an unfavourable early comparison may be an index configuration finding rather than a model finding, and telling those apart takes a deliberate protocol.

There is also a floor under how fast this can go that has nothing to do with money. Nine months sounds generous against a thirty-day backfill until the sequence is laid out: the evaluation has to precede the build, the build has to precede the cutover, and each stage produces a decision that the next one depends on. Starting in month six leaves no room for the case where the comparison comes back unfavourable and the plan has to change.

The premise underneath all of it#

Every number in that table is spent on an assumption: that a model scoring four points higher on public benchmarks retrieves better on this corpus, for these queries, against these judgements. Public retrieval benchmarks are built from web pages, academic abstracts and news; a corpus of maintenance manuals in three languages with heavy internal jargon resembles none of them, and a model can win by four points on the benchmark and lose on the corpus, particularly where the old model has been in place long enough that chunking, prompt prefixes and ef_search were all tuned around its behaviour.

Which makes the first bar of that chart the load-bearing one. Month one of a nine-month clock is not for provisioning capacity or negotiating quota; it is for finding out whether the premise holds, with a protocol decided in advance, because a capacity commitment made before that answer exists is a commitment you cannot walk back from cheaply.