Translate the query or embed it natively: each path loses something

A cross-lingual retrieval system has two architectures available to it, and they differ in where the language boundary is crossed. The translation path crosses it in a separate service before retrieval begins: a machine translation (MT) system turns the Spanish query into English, and everything downstream is an ordinary monolingual English search. The native path crosses it inside the encoder, by relying on a multilingual model to place a Spanish query vector near the English documents that answer it — the alignment property from chapter 1.

Both paths lose information, and they lose different information for different reasons. What follows works the comparison on one concrete system: a corpus of 220,000 English documents — contracts, filings, and counsel memoranda — and users who are Spanish-speaking lawyers at Madrid and Bogotá offices, typing in Spanish because that is the language they think about the law in.

Where each path loses a term#

One query in that system's evaluation set reads ¿las arras penitenciales se pierden si el comprador desiste? A commercial translation service renders it as "are the penitential deposits lost if the buyer withdraws?", which is not English and not law. Arras penitenciales is a Spanish civil-code instrument with no common-law equivalent; the nearest English concept is an earnest-money deposit with a contractual right of withdrawal, and the phrase a US associate would have written in the memo that answers this question is "liquidated damages for buyer's termination".

Embedding the Spanish query directly does not solve that either. The multilingual encoder maps arras to a region of the space near deposit and down payment, which retrieves eleven documents about escrow and none about the specific remedy.

The two failures have the same root and different visibility, and the difference in visibility is what most of the rest of this chapter turns on. In both cases a term of art in one legal system has no counterpart in the other, so no single word or vector can carry it. In the translation path the failure is a string: "penitential deposits" is sitting in a log, a lawyer can read it, say what it should have been, and the correction is a dictionary entry. In the native path the failure is a position in a space of several hundred dimensions. There is no artefact, nothing to read, and no lever short of changing the encoder.

The MT hop buys one thing that is easy to undervalue: it lets the retrieval side be entirely English, which means the strongest available English encoder, the strongest English reranker, an English lexical branch that actually contributes, and a query string a human can read in the logs. Each of those follows from the same step. Once the query is English, the retrieval problem has no cross-lingual component left in it, so every component may be chosen from the much larger and much stronger pool of English-only ones — and the lexical branch, which chapter 3 showed is structurally incapable of matching across languages, starts working instead of merely voting for same-language documents. Everything that is hard about cross-lingual retrieval gets pushed into one service whose output is text, and text can be inspected, corrected and tested.

The native path buys a shorter, cheaper, more reliable request path, and it removes an entire failure domain. It also handles inputs the translation path mangles — code-switched queries where a Spanish sentence carries English contract terms, which is how these lawyers actually write, since half the agreements they work on are in English.

The comparison has to run on native queries#

One condition has to hold for the comparison to mean anything, and it is the trap chapter 2 set up. The queries must be native Spanish, written or rewritten by Spanish speakers.

Follow what happens if they are not. Suppose you build the evaluation set by machine-translating your existing English queries into Spanish. The translation path then takes those Spanish sentences and translates them back into English — so it is being handed text that is one round trip away from the English it is about to be turned back into, and MT round trips are close to lossless on sentences that MT itself produced. The native path gets no such help: it has to place that Spanish text near English documents on the strength of alignment alone. The comparison is therefore circular, it is the default thing to reach for because the English set already exists, and it reliably overstates the translation path by five to ten points of nDCG.

Run both over the same 200 native Spanish queries, judged against the same English documents at the canonical level — the convention from chapter 3, where a judgment names the content rather than one language's rendering of it. One of the rows below combines the two paths with reciprocal rank fusion (RRF), which merges two ranked lists by giving every document a score of one over its position in each list and adding those scores up. A document at position two in one list and position three in the other therefore scores one half plus one third, while a document only one path liked scores one half and nothing, so documents both paths ranked well finish above documents only one of them liked.

Read the first and last rows together before reading anything in between, since they bracket everything else. The native path scores 0.581 and an English query against the same English corpus scores 0.782, so the whole cost of the language boundary on this system is about two tenths of nDCG. Every other row is a partial recovery of that gap. The fusion row at 0.729 has closed roughly three quarters of the distance from the native path to the ceiling, which is a more useful description of it than "the highest number in the table".

Latency is the operational cost that bites. The 520 ms at p95 — the delay the slowest one query in twenty sees, against 180 ms for the median at p50 — lands on top of a retrieval budget that was probably 300 ms, and it arrives from a network dependency whose tail is not under your control.

The translation cost is real and small, and it is worth deriving rather than quoting because the derivation is what transfers to your own vendor. The Spanish query at the top of this chapter is about 60 characters, which is typical, so 100,000 queries is about six million characters. At roughly 60 characters per query and $20 per million characters, the line in the table reads about $12 to translate 100,000 Spanish queries, which is under a thousand dollars a year at this volume and will not be the deciding factor. Run the multiplication against your own vendor's per-million rate before carrying that number across, because the character count is the part of this that travels and the unit price is the part that does not.

Reading the losses rather than the averages#

The per-query comparison is more informative than the eight-point gap between the first two rows. Of the 200 queries, the MT path beat the native path on 118, the native path won on 42, and 40 tied.

Those three counts sum to 200 exactly, which makes them a complete partition of the evaluation set rather than three interesting observations, and that is what licenses the reading that follows. The eight-point mean gap is the net of a two-sided distribution: MT does not beat the native path on most queries by a little, it loses outright on more than one query in five. An architecture decision made on the mean alone is discarding the 42 queries where the rejected path was better, and those 42 are not randomly distributed.

Sorting the 42 native wins by what they had in common produced two recognisable groups: 19 were code-switched, where the translation service tried to translate the English contract terms back into Spanish and then into different English words, and 14 were queries where the Spanish phrasing carried a distinction the English translation flattened. Those two groups account for 33 of the 42, which leaves nine the exercise could not classify — worth stating rather than rounding away, because an unexplained remainder that size is the difference between "we understand this failure mode" and "we understand most of it".

The 118 MT wins were mostly unremarkable — ordinary legal questions where an English-only encoder is simply better than a multilingual one at English retrieval, which is what the benchmark gap between the two model families would predict. Among the queries where MT lost badly, 31 of the worst involved civil-law terminology with no common-law equivalent, and that group is addressable, which is the most useful finding in the whole exercise.

GLOSSARY = {
    "arras penitenciales": "liquidated damages for buyer's termination",
    "usufructo": "usufruct (life interest in property)",
    "sociedad limitada": "limited liability company (S.L.)",
    "recurso de amparo": "constitutional appeal (recurso de amparo)",
}
DO_NOT_TRANSLATE = ["S.L.", "S.A.", "BOE", "LSC"]

def translate_query(es: str) -> str:
    en = mt_client.translate(
        text=es, source="es", target="en",
        glossary=GLOSSARY, adaptive=False,
        do_not_translate=DO_NOT_TRANSLATE,
    )
    return en

def spanish_query_vector(es: str, budget_ms: int = 400):
    try:
        return ENGLISH_ENCODER.encode(translate_query(es), timeout=budget_ms)
    except (TimeoutError, MTServiceError):
        metrics.incr("mt.fallback")
        return MULTILINGUAL_ENCODER.encode(es)

Latency has a partial answer that is worth applying before the architecture argument gets heated. Query text repeats more than people expect — on this corpus, 38% of Spanish queries in a month were exact repeats of an earlier query after normalisation — so a translation cache keyed on the normalised query string serves that share at under 5 ms and pulls p95 down from 520 ms to about 340 ms, with the tail now dominated by first-time queries. The cache also gives you a place to pin corrections: a term the glossary handles badly can be fixed by writing the right English into the cache entry for that exact query, which is crude and effective for the fifty queries that matter most.

Choosing, and what the choice costs#

The conditions that made translation the right default there are worth naming, because they are what the decision generalises over: one corpus language, a specialised domain, terminology that experts can enumerate, moderate query volume, and a latency budget with 500 ms of headroom. Taking the fusion row instead is a genuinely defensible answer wherever that budget is wider.

The reasoning does not generalise, and the conditions that flip it are specific. A mixed corpus is the strongest one: with documents in five languages, "translate the query into the corpus language" has no single answer, and translating into all five multiplies latency and cost by five while creating exactly the ranking competition chapter 3 described. High query volume with a hard latency budget flips it, since the translation hop is unavoidable and the native path is free. Fourteen locales flips it partway, because fourteen language pairs each need their own glossary and their own quality monitoring, and glossaries are maintained by people who have other jobs. And a corpus where users routinely code-switch flips it, as the 19 queries above did.

The third column is the option teams forget, and it is sometimes the best one. Price it the same way. Translating 220,000 documents at 2,000 words each is four hundred and forty million words, and at the average length of an English word with its trailing space that is roughly 2.6 billion characters — about $52,000 at commercial per-million rates, or a few thousand dollars of GPU time with an open translation model, paid once and amortised over every query. Notice that this is the same per-character arithmetic as the query hop, run over the corpus instead: what changes is that the corpus is finite and the query stream is not, so the query path pays forever and this one pays once. It moves the whole problem off the request path and into ingestion, where latency does not matter and a human can review samples. What it buys in quality it charges back in the maintenance burden chapter 3 documented, since every translated document is a rendering that can go stale and compete with its source, and the canonical-id discipline becomes mandatory rather than advisable.

Monitoring the path you did not choose#

Committing to one path means committing to monitor the failure mode of the one you did not pick, since that failure mode is now the one that will surprise you.

Having chosen translation, the thing that breaks is the translation service, and it breaks in two ways. The loud way is an outage or a latency spike, which the fallback handles and which shows up as a jump in mt.fallback — worth alerting on, because a sustained fallback means the system has silently switched to a configuration scoring 0.58 instead of 0.70. The quiet way is a model update on the vendor's side that changes how a term is rendered, which moves retrieval quality with no deploy on your side; a weekly re-run of the 200-query set against the fixed judgments is what catches it, and it costs $0.02 of translation.

Had the native path been chosen, what breaks is everything the earlier chapters measured, and it breaks without producing an artefact anyone can read. The alignment deficit is fixed until you change encoders, terminology errors have no log line, and the only instrument is the per-cell evaluation from chapter 4 run often enough to notice drift. The argument that native embedding has "fewer failure modes" is worth naming as the specific error it is: it counts services, and the number of services is genuinely smaller. The quantity that matters is the number of ways an answer can be wrong, and that number is not smaller — it is the same set of terminology and alignment failures, minus the ones MT introduces, plus the loss of every artefact that would let you find them. A system with one service and no observability into its errors is not the safer of the two.

Where this is heading is worth knowing even though it is out of scope here. The strongest current results in cross-lingual retrieval come from putting a multilingual cross-encoder reranker — a model that reads the query and one candidate document together and scores the pair, instead of embedding each separately and comparing vectors — behind a native first stage, which recovers much of the gap without a translation service, and from using an LLM for query translation with the domain glossary in the prompt rather than a dedicated MT system, which handles code-switching and terminology in one step at higher latency. The survey Bridging Language Gaps: Advances in Cross-Lingual Information Retrieval with Multilingual LLMs covers both lines and the benchmarks people report them on; Cross-lingual retrieval: search across languages, explained is a shorter route to the intuition about shared spaces against translate-then-search.

Neither path is free, and the choice between them is a choice about where you would rather the losses be. Translation pays money and tail latency, adds a service that can fail, and returns every one of its errors as a readable string that a domain expert can correct in an afternoon. Native embedding pays nothing, adds nothing, and returns its errors as coordinates. On a single-corpus system with enumerable terminology and latency headroom, the readable errors are worth the half-second; change any of those three conditions and the arithmetic changes with it.

None of that changes the procedure. Take the queries your users actually type in their own language, fix the judgments at the canonical level, run both paths over the same set, print the per-cell table with counts beside every number, and read the queries where the paths disagree. The result is a decision you can defend, plus a list of the specific terms your system does not understand — and that list, more than the architecture, is what the next quarter of work is made of.