Matryoshka embeddings, dimension truncation, quantized and binary embeddings

Read these first

• Embedding models

What you will know

• Truncate an embedding dimension and measure the quality loss

  A model trained for nested representations can preserve useful signal in prefixes, whereas arbitrary truncation may destroy nearest-neighbor ordering.

• Calculate the storage saving at your corpus size

  Include vector bytes, metadata, replicas, and index overhead; the raw dimension ratio is not the bill you pay.

• Compare float, int8, and binary embeddings on the same queries and index

  Integer and binary formats can require different distance implementations, normalization, and hardware paths, so nominal byte savings do not predict recall or latency.

• Choose a quantisation level with a measured recall floor

  Validate against representative queries and hard negatives, because average recall can hide regressions in the queries users care about.

• Make the storage/quality trade explicit rather than accidental

  Record the selected format, dimension, evaluation result, and rollback threshold so an index rebuild does not change behavior invisibly.

Chapters

1. 1

   Matryoshka training makes every prefix a usable representation on its own

   The objective is applied at several nested dimensions at once, so the first 256 components of a 1536-dimension vector are themselves a trained embedding rather than an arbitrary slice. Cut an ordinary model the same way and you keep whichever components happened to land first, destroying neighbour ordering in ways that a spot check of one query will not reveal. Truncation is a model property, not an operation you can apply to any vector.

   Read chapter
2. 2

   A truncated prefix has to be re-normalised before anything compares it

   Slicing a unit-norm vector leaves a shorter vector whose length now varies from document to document, so a dot-product or L2 index quietly starts ranking partly by magnitude instead of by direction. Re-normalise every prefix, and apply exactly the same truncation and normalisation to the query vector and to the stored ones, because a mismatch returns plausible neighbours and raises no error anywhere. Skip this and the quality loss you measure is partly your own bug — and the next chapter offers an explanation that will happily absorb it.

   Read chapter
3. 3

   Truncation costs you deep ranks long before it touches the top ones

   Dropping 1536 to 256 typically leaves recall@10 almost untouched while recall@100 slides, because the prefix carries coarse topical structure and the discarded tail carried the fine distinctions that separate rank 40 from rank 90. Whether that matters depends on what consumes the list: serving ten results directly is safe, while feeding a hundred candidates into a reranker or a selective filter is not.

   Read chapter
4. 4

   int8 and binary change the distance kernel, not merely the byte count

   Quantisation needs a scale calibrated from your own vector distribution, a normalisation convention that both sides agree on, and a distance implementation that matches the format — Hamming for bits, a dequantising or integer path for int8, possibly a different hardware kernel. A store that accepts the smaller vectors and silently keeps its float distance will return plausible, wrong neighbours, so recall must be checked against exact float search.

   Read chapter
5. 5

   Binary embeddings work because something rescores what they return

   The near-full-precision claim is not about one-bit vectors being accurate; it is a two-stage design where Hamming search over the binary index oversamples a few hundred candidates and full-precision vectors rescore them. The trick breaks where you assumed it would not: you still store the float vectors somewhere and pay to fetch them, the oversampling factor is a tunable that decides your real recall, and anything stage one missed can never be recovered.

   Read chapter
6. 6

   The bill is not the dimension ratio, at 50M chunks least of all

   Halving 1024 float32 dimensions removes 2 KB per chunk, which sounds like a hundred gigabytes until you count what sits beside it: metadata payloads and stored text, graph links in the index, replicas multiplying all of it, and snapshots. Compute the saving against the actual line items and order the options by risk — truncation, int8, binary with rescoring — before promising finance a number.

   Read chapter
7. 7

   A recall floor on hard queries gates the change, and the record makes it reversible

   Average recall over a friendly query set will approve almost any compression. Set the criterion on the queries that hurt — near-duplicate candidates, hard negatives, rare entities — and require a floor rather than a mean. Then record the chosen format, dimension, evaluation result and rollback threshold with the index itself, so the next rebuild cannot quietly change retrieval behaviour with nobody able to say what moved.

   Read chapter