Index maintenance

Read these first

• Vector databases and selection criteria

What you will know

• Add, update, and delete documents in a live index

  A delete flag can hide a record from one query path while stale vectors remain reachable through another, especially when replicas lag.

• Re-embed a corpus after a model change without downtime

  A model change can alter vector dimensionality or neighborhood geometry, so old and new representations cannot safely share one namespace.

• Run two index versions in parallel and cut over

  The cutover needs atomic routing and a rollback point; otherwise mixed generations make relevance regressions difficult to attribute.

• Estimate the cost and duration of a full reindex before committing

  Embedding throughput, provider pricing, storage amplification, and throttling determine whether a seemingly affordable migration misses its maintenance window.

• Measure tombstone accumulation and recall degradation over repeated delete cycles

  Tombstones consume search and storage budget until compaction, and recall can decay long before capacity alarms fire.

Chapters

1. 1

   A delete hides the row while copies of it stay reachable elsewhere

   Upserts and deletes on a live index are several writes that commit at different times and to different places. A soft-delete flag honoured by one query path leaves the vector answerable through another; a read replica serves the old row for seconds or minutes; and the content also sits in result caches, embedding caches, prompt logs, derived summaries and last night's snapshot. Enumerate those copies before you claim a document is gone.

   Read chapter
2. 2

   A shortened revision leaves its old chunks behind, unreferenced and fully retrievable

   Edit a document and it re-chunks into a different number of pieces: upserting chunks 1..n overwrites what it can and leaves chunks n+1..m from the previous revision sitting in the index, unreferenced by the writer and returned by every query. Nothing errors and nothing is tombstoned, so neither a tombstone gauge nor recall against exact search can see it — the exact search finds the same orphans. Treat an update as a reconciliation: enumerate the chunk ids the new revision produces, diff them against the ids the index holds for that document, delete the difference, and carry a document-level revision key so a stale chunk is identifiable at query time.

   Read chapter
3. 3

   Tombstoned vectors keep spending search budget until compaction runs

   Deleting from a graph index marks nodes rather than removing them, so traversals still walk through them and memory stays where it was. Delete 30% of a corpus over six months without a rebuild and you are paying full price for a two-thirds-sized index while recall drifts down. Monitor the tombstone ratio and periodic recall against exact search, because capacity alarms fire long after quality does.

   Read chapter
4. 4

   Two embedding generations cannot share one namespace, even at equal dimensionality

   Two models producing 1024-dimension vectors do not produce comparable ones; each defines its own space, and a distance computed across generations is arithmetic without meaning. Mix them in one index and nothing errors — queries return results, latency is normal, and the ranking is quietly scrambled in proportion to how much of the corpus has been migrated. Re-embedding is therefore a second index, never an in-place update.

   Read chapter
5. 5

   Sustained embedding throughput, not the token price, sets the reindex date

   Eighty million chunks times a per-token price gives a bill you can approve in an afternoon; the schedule is decided by what the provider will actually let you push per minute after rate limits, retries and throttling, and by whether your store can ingest at that rate while serving traffic. Compute five quantities before naming a date: spend, sustained throughput, storage amplification while both copies exist, the window you have, and the catch-up tail for everything written while the backfill runs. The last one is the one people forget, and it is why a backfill never finishes where it started.

   Read chapter
6. 6

   The corpus keeps changing for every day the backfill is running

   A reindex measured in days is not a batch job over a static corpus. Documents are inserted, edited and deleted throughout, and an index built from a snapshot is missing every one of those writes by the time it completes. Either dual-write to both generations from the moment the backfill starts, or keep a change log and replay every insert, update and delete since the snapshot — and budget that catch-up as part of the schedule, because on a corpus with real churn the tail is what turns a long weekend into two. Before any cutover, gate on freshness: newest document timestamp on both sides, and per-document chunk count parity on a sample.

   Read chapter
7. 7

   Two fully built versions with atomic routing make a regression attributable

   Build the new index to completion beside the old one, keep both queryable, and move traffic with a single routing switch rather than a gradual in-place migration. That gives a rollback point and, more importantly, an experiment: any relevance change is attributable to one variable, whereas a half-migrated index blends generations and makes the question unanswerable. Verify counts, freshness and top-k overlap before the switch, not after — counts drift slowly and a frozen query set only exercises content that predates the backfill, so the freshness gate is the check that will actually catch a bad cutover.

   Read chapter
8. 8

   Six months of drift with no deploys is a maintenance failure, not corpus growth

   Work it in order: tombstone ratio and delete volume first, then orphaned chunks from documents that were edited, then recall against exact search on a frozen query set, then the graph's degradation since the last rebuild, then how the 20% of new documents changed length and topic distribution. Raising k rationalises the symptom. What should have existed from day one is a recall-versus-exact check and a tombstone gauge on the same dashboard as latency.

   Read chapter