Asymmetric means role, not length, and the card names the strings

Asymmetric search is retrieval in which the two texts being compared are playing different parts: one of them is a question somebody typed into a search box, the other is a passage that might answer it, and a model trained for the job has learned to place the two near each other. Because one set of weights encodes both sides, the model has to be told which part the text in front of it is playing, and the telling is done with a prefix — a short literal string, query: or passage: in the case of the E5 family of encoders, glued to the front of the text before it reaches the encoder. An ingestion job that puts query: on a support runbook is stating that the runbook is a question, and the model then places it where it places questions.

The arrangement exists because a search system spends its life comparing things that are not alike. A person asking how to rotate a credential writes a handful of words that share very little vocabulary with the paragraph that answers them, so a model that only knew how to tell whether two texts resemble each other would rank the pages that repeat the question's words above the page that resolves it. Training on question-and-answer pairs is what closes that gap, and the prefixes are how the finished model keeps the two roles straight afterwards. Where an engineer meets all of this is a line in the model card, the README published alongside a model's weights, naming the two strings and saying which side each belongs on, and thereafter as two string constants: one in the code that indexes documents, one in the code that serves searches. The rest of this topic is about those two constants being correct, applied once each, and applied to the right side.

What the training objective actually rewards#

E5 and models like it were trained on pairs where one side is a question and the other is a passage that answers it: MS MARCO contributes on the order of 500,000 real search queries against a corpus of 8.8 million web passages, Natural Questions contributes Google queries paired with Wikipedia paragraphs, and there are several more of the same shape.

It is worth working through what that objective does to the geometry, because the prefix only makes sense once you can see what the model was being paid to do. Training proceeds in batches. A batch is some number of question-and-answer pairs loaded together, and for each question in it the objective has two terms. The first pushes the question's vector toward the vector of its own answering passage. The second pushes it away from the vector of every other passage in the same batch — the in-batch negatives, so called because they cost nothing to obtain: they are already loaded, and every passage that is not this question's answer is by construction a wrong answer for it.

Now ask what a model has to learn in order to score well under that second term. The other passages in a batch are not random noise. Sampled from a web corpus, several of them will be about the same broad subject as the question, because subjects repeat. So topical similarity cannot be the feature that wins the objective — every passage in the batch that shares a topic with the question would score high, and the true answer would not separate from them. The only feature that reliably separates the true answer from a topical near-miss is the relation this passage resolves this question, and that is the relation the weights end up encoding. Repeat across hundreds of millions of pairs and the encoder arrives at a geometry in which "text that asks about X" and "text that answers about X" sit close together, closer than "text that asks about X" and "text that merely mentions X".

Nothing in that objective mentions length. It happens that questions in those datasets are short and passages are long, so length correlates with role in the training data, but the relationship the model learned is between a question and its answer. A 300-word support ticket describing a symptom in detail is still playing the query role, and a 12-word glossary entry is still playing the passage role, and the model expects to be told which is which regardless of how many tokens each one runs to.

Two towers, then one tower and a switch#

The mechanism that carries that information came out of an earlier design. Dense Passage Retrieval, the 2020 system that made dense dual-encoder retrieval practical for open-domain question answering, used two physically separate encoders built on BERT, the transformer text model most retrieval work of that period started from — one set of weights only ever saw queries, the other only ever saw passages. That works, and it doubles your parameter count, your GPU memory at serving time, and the number of checkpoints you have to keep in sync.

The step worth naming there is that the two-tower arrangement never needs to be told anything. A tower that has only ever seen queries cannot be handed a passage by mistake in the architecture's own terms — the role is carried by which function you called, and a function is a thing a type system and a code reviewer can both see. Collapsing the towers is a design decision rather than a requirement, taken to halve the serving cost, and what it removes is exactly that carrier. The role still has to arrive somewhere, so it moves into the only channel a single encoder has: the input text itself. The E5 and BGE families collapsed the two towers into one set of weights and recovered the role distinction with a short literal string prepended to the input. The prefix is the surviving remnant of the second tower. When you drop it, you are not omitting a formatting nicety; you are asking one encoder to do the job of two without telling it which job it is doing on this call.

That framing explains the shape of the failure. With query: on both sides, every document is embedded into the region the model reserves for questions. Queries still land near documents that are topically related, because topic survives the mistake — a question about API keys and a document about API keys share most of their content words either way. What does not survive is the finer signal that separates the document that answers this from the document that is merely about this, which is the signal the objective above spent hundreds of millions of pairs building, and it is the signal that decides ranks 1 through 10.

The strings are model-specific and guessing costs you recall#

There is no convention here. Each family chose its own tokens, chose independently whether the document side gets one at all, and in some cases changed the answer between versions. The exact strings, including whitespace, are in the model card, and reading it is the only reliable way to get them.

Two rows in that table are worth dwelling on. BGE puts an instruction on the query and nothing on the document, which means a pipeline that symmetrically adds "the prefix" to both sides is wrong for BGE in a way it is not wrong for E5. And bge-m3 removed the query instruction that bge-large-en-v1.5 requires, so a team that upgraded within the same family and kept its formatting code untouched carried a stale instruction into a model that was never trained with it. The intfloat/multilingual-e5-large model card on Hugging Face is unusually direct about this and states plainly that the prefixes are required for both symmetric and asymmetric use; the BGE write-ups, including State-of-the-Art BGE embeddings for Retrieval Augmented Generation, spend most of their formatting discussion on the query instruction because that is where BGE's asymmetry lives.

In code, the whole mechanism is this small:

from sentence_transformers import SentenceTransformer

model = SentenceTransformer("intfloat/multilingual-e5-large")

QUERY_PREFIX = "query: "
PASSAGE_PREFIX = "passage: "

def embed_query(text: str) -> list[float]:
    return model.encode(QUERY_PREFIX + text, normalize_embeddings=True).tolist()

def embed_passage(text: str) -> list[float]:
    return model.encode(PASSAGE_PREFIX + text, normalize_embeddings=True).tolist()

# The strings that actually reach the tokenizer:
#   "query: how do I rotate an API key"
#   "passage: To rotate a credential without downtime, first create a second key..."

The trailing space matters, because query:how and query: how tokenize differently and only one of them matches what the model saw in training. The step underneath that is worth spelling out. A subword tokeniser splits text into pieces drawn from a fixed vocabulary, and in the SentencePiece scheme these models use, a leading space is part of a piece rather than a separator between pieces — which is why the tokens printed in later chapters carry a ▁ mark on their front. Delete the space and the colon merges with the first letters of the user's word into a piece the model has essentially never seen in that position. The difference is small — in one internal comparison on the same 480 queries, dropping the space cost about two points of recall@10 rather than forty — but it is a free two points, and it is the kind of detail that gets lost when someone reformats a constants file.

Model cards are not uniformly good at making this findable. On some repositories the prefixes appear in a prominent "Usage" section with a warning; on others they exist only inside an example snippet halfway down, where a reader skimming for the dimension count and the context window will pass straight over them. A few document the strings in a section about one task — semantic similarity, say — and leave a reader who came for retrieval to infer that the same rules apply. When you are checking a card, look for four specific things: the literal strings including whitespace, which side each belongs on, whether the model expects normalized embeddings, and whether the card distinguishes between task types. If the card is silent on prefixes and the model is recent, silence usually means none are needed, and you can confirm it in thirty seconds by embedding the same sentence with and without a candidate prefix and checking whether the cosine between the two results is close to 1.0 — a model that was never trained with prefixes treats them as ordinary content and shifts modestly, while a model that was trained with them has a much sharper response to the exact string.

The other thing worth pulling out of the card is whether the family defines roles beyond the two. Nomic's model exposes four, because the same weights were trained to serve retrieval, clustering and classification, and each of those tasks wants a different arrangement of the same corpus. A pipeline that indexes documents under search_document: and then runs k-means over those vectors is using retrieval-shaped geometry for a clustering job, which is a smaller mistake than mixing up query and passage but is still the same category of mistake: asking for one arrangement and using another.

Why the wrong version raises nothing#

A prefix is text. The tokenizer will encode passage: in front of a user's question as cheerfully as it encodes it in front of a document, produce a valid sequence, and hand back a well-formed 1024-dimensional vector with unit norm.

The reason every safeguard stays quiet is worth stating as a property rather than as a list, because the same property recurs throughout this topic. Each of those safeguards is a check on a quantity that does not depend on the prefix. Dimensionality is fixed by the architecture. Unit norm is imposed after the fact by the normalisation step. Cosine range follows from unit norm. Latency depends on token count, which barely moves for two extra tokens. A check whose value is identical whether the formatting is right or wrong cannot discriminate between the two, however carefully it is written, and every downstream check that a careful team actually runs is of that kind: the vector has the right dimensionality, the norm is 1.0, the cosine similarities are in range, the index accepts the upsert — the insert-or-replace write a vector store takes — latency is unchanged, and no exception is thrown anywhere in the stack. The only observable that does vary with the prefix is where the vector sits, and the only thing that reads a vector's position is retrieval quality — which is exactly the thing that is hard to eyeball, because wrong-but-topical results look like the search engine is working and merely having a bad day on this query.

This is what makes the defect expensive relative to its size. A misconfigured similarity metric usually announces itself — cosine scores clustering weirdly, or an ordering that is visibly inverted. A truncation bug shows up as a length correlation you can plot. Prefix errors produce a smooth, uniform, plausible degradation across the whole corpus, and the natural human response to plausible-but-mediocre search results is to conclude that the embedding model is not strong enough for your domain. Multilingual vector search with the E5 embedding model reports the same pattern from the other direction: applying the prescribed prefixes consistently is worth a substantial accuracy jump over running the same model without them, on the same data and the same index.

Where the word misleads#

The most common wrong reading of "asymmetric" treats it as a statement about length, which leads to the conclusion that it is a chunking concern — make the chunks smaller and the mismatch goes away. Chunk size is a real parameter with real effects on retrieval, and it has nothing to do with this. If you chunk your corpus down to single sentences so that documents and queries are the same length, an E5 model still needs query: on one side and passage: on the other, because the roles are unchanged. Conversely, a long verbose query and a long document both need their own role markers even though their lengths match.

Asymmetric formatting also does not mean that every embedding model needs it. Models trained on semantic textual similarity — the all-MiniLM and all-mpnet lines, most of the older sentence-transformers catalogue — were trained on pairs that play the same role on both sides, such as two paraphrases of one sentence. Run the batch objective above with that training data and the second term never has to separate an answer from a topical near-miss, because there is no answer relation in the data at all. Those models have no prefix in the card because none was in the training data, and prepending one adds tokens the model has no learned handling for, which shifts the vector slightly and can only hurt. Copying a formatting convention from one model to another is its own bug.

The prefixes are also not instructions in the sense that a language model prompt is an instruction, and this is the belief most likely to survive the rest of this topic if it is not named now. It is a tempting belief because the strings are readable English: passage: looks like a note to the model, so it feels as though a synonym would do. The model does not read passage: and reason about what it means. It is a fixed token sequence that co-occurred with one side of every training pair, and its effect is entirely learned association, which is why the exact string matters and why paraphrasing it into document: degrades quality instead of being understood. The instruction-tuned embedding models — Instructor, and the more recent E5-Mistral line built on decoder LLMs — do accept freeform task descriptions and generalise across them, which is a genuinely different mechanism and one that is still moving quickly.

The word "asymmetric" therefore describes the model's training rather than your data, and the whole of what it asks of you is two constants. One set of weights was taught a relation between a question and its answer, the second tower that used to carry the role was removed to save memory, and a literal string took its place. A text arriving without that string, or with the other one, is not malformed — it is a well-formed statement that the text is something it is not, and the encoder will act on the statement.

You now have the two strings for your model, and the knowledge that they belong on different sides. Where they get applied is the next question, and it is not symmetric either: one of the two is evaluated on every request and can be changed with a deploy, while the other is frozen into 4.2 million stored vectors the moment the index is built.