Masking every flag strips the tokens that distinguish one chunk

The redaction change went into the ingestion pipeline on a Tuesday and the support assistant was noticeably worse by Thursday. The complaint from the pilot team was that it answered account-specific questions with information about other customers' tickets, confidently and constantly. The retrieval eval set — 180 questions of the form "what did we tell Northwind about their gateway timeouts" — had gone from recall@5 of 0.78 to 0.31, and mean reciprocal rank from 0.64 to 0.19. Recall@5 is the share of those questions whose correct passage appeared anywhere in the top five results, and mean reciprocal rank averages one over the position of the first correct result, so it falls faster when the right passage slides from second place to eighth. Nothing about the retriever, the embedding model, or the chunking had changed. The only change was that person names were now masked before embedding.

The pipeline those numbers describe is the ordinary one: each document is split into chunks of a few hundred words, an embedding model turns each chunk into a vector of a few hundred numbers positioned so that passages about similar things land near one another, and a question is answered by embedding it the same way and returning the nearest chunks. Whatever survives redaction is what the vector is computed from, which is why a change to the redactor is a change to the retriever.

Pull two chunks out of the index and the mechanism is visible immediately.

before (chunk 4471, ticket NW-2210)
  Priya Raman escalated on behalf of Northwind Logistics: their Atlas Gateway
  cluster returns ERR_TLS_HANDSHAKE for roughly 4% of requests since the
  certificate rotation on 12 March.

before (chunk 9930, ticket BR-1187)
  Tomás Oliveira escalated on behalf of Braid Retail: their Meridian Sync
  workers stall at 200 queued jobs since the migration on 3 April.

after (chunk 4471)
  [REDACTED] escalated on behalf of [REDACTED]: their [REDACTED] cluster
  returns [REDACTED] for roughly 4% of requests since the certificate
  rotation on 12 March.

after (chunk 9930)
  [REDACTED] escalated on behalf of [REDACTED]: their [REDACTED] workers
  stall at 200 queued jobs since the migration on 3 April.

The cosine similarity between those two masked chunks is 0.94, against 0.61 before the change. Cosine similarity is the usual closeness score between two embeddings, running from 0 for vectors pointing in unrelated directions to 1 for vectors pointing the same way, and 0.94 means the index can barely tell the two chunks apart. Every token that distinguished one customer's problem from another's has been replaced by the same string, and what remains is boilerplate shared by most escalation tickets in the corpus. A query mentioning Northwind has nothing in either chunk to prefer, so the retriever falls back to whatever the residual noise favours, which is effectively arbitrary.

The detector was not malfunctioning. Northwind Logistics is a proper noun in an organisation-shaped position, Atlas Gateway is a two-token capitalised phrase, and a general-purpose named-entity recognition model — an NER model, the standard component that tags spans of text as people, places, organisations and the like — asked for person and organisation entities will assign both a respectable score. ERR_TLS_HANDSHAKE was flagged by a generic identifier rule tuned to catch reference numbers. All three were false positives, all three were replaced by the same placeholder, and the placeholder is what did the damage: three distinct entity types collapsed into one token, and repeated across 214,000 chunks, the corpus lost most of its variance.

Placeholders that keep documents distinguishable#

The repair has two parts, and taking only the first is common.

Typing the placeholder restores the syntactic and semantic role of the removed span. [PERSON], [ORG], [EMAIL], [DATE] tell both the embedding model and the generating model what kind of thing sat there, which is enough for a summariser to write "the customer" rather than losing the sentence structure. Typing alone does not fix the retrieval problem, because every organisation in the corpus is still [ORG].

Making the placeholder stable per entity is the part that restores distinguishability. Within a document, every occurrence of the same underlying value maps to the same surrogate, so coreference survives — coreference being the property that two mentions in a text point at the same real thing: [PERSON_1] escalated ... [PERSON_1] confirmed the fix still says that one person did both things, where [REDACTED] escalated ... [REDACTED] confirmed does not. Across documents, a keyed derivation gives the same value the same surrogate everywhere, so all of Northwind's tickets share a token and an account-scoped query has something to match on. The derivation is an HMAC — a hash-based message authentication code, which is an ordinary hash function combined with a secret key so that nobody without the key can reproduce the output for a value they are guessing at.

import hmac, hashlib

SURROGATE_KEY = load_secret("pii/surrogate-key")   # rotated, never in the corpus

def surrogate(entity_type: str, value: str) -> str:
    normalised = " ".join(value.casefold().split())
    digest = hmac.new(SURROGATE_KEY, f"{entity_type}:{normalised}".encode(),
                      hashlib.sha256).hexdigest()[:10]
    return f"[{entity_type.upper()}_{digest}]"

Applied to the two chunks above, Northwind Logistics becomes [ORG_1f4b9c72a0] in every ticket it appears in, Priya Raman becomes [PERSON_8e02d61aa5], and the cosine similarity between the two masked chunks falls back to 0.66. Retrieval recall@5 on the same 180-question eval set recovered to 0.74 — short of the 0.78 baseline, because a surrogate token is still a worse query match than the literal string a user types, and close enough to ship.

The rotation and storage of SURROGATE_KEY is the whole security argument for this design. A surrogate derived with a secret key is a pseudonym — a stand-in value that hides the original while still allowing anyone with the key, or with a good enough guess, to link it back — and a pseudonym under GDPR is still personal data, so the mapping is exactly as sensitive as the values it stands for. Losing the key does not by itself reverse anything, since SHA-256 is one-way, but the value space for many entity types is small enough to enumerate — every Dutch postcode, every date of birth in a 90-year range, every name in a national registry — so an attacker with the key and a candidate list can invert the surrogates by brute force in minutes. That is why the key never travels with the corpus and why the surrogate should be truncated rather than full-length, since a shorter digest deliberately collides.

The bottom row deserves a caution. Replacing Priya Raman with a plausible invented name such as Elena Sáez keeps the text completely fluent, which measurably helps models that were trained on natural prose and degrade on bracket-heavy input. It also produces documents that read as true and are not, so a support agent seeing a retrieved chunk may act on a name that belongs to nobody, and an unlucky draw may produce a name that belongs to a real and unrelated customer. Microsoft Presidio's anonymizer ships operators for all six rows of that table, which makes the choice a configuration decision rather than an implementation project, and the configuration is the part worth arguing about.

What raising recall everywhere actually costs#

The instinct after a bad recall report is to lower thresholds until every entity type clears 99%. The cost of that is measurable, and measuring it before the argument is what turns the discussion from a values debate into a trade.

Sweeping the person-name detector's score threshold over the same 400-transcript corpus and the same 180-question retrieval eval gives this:

The last column is a blind pairwise human preference between summaries generated from the corpus at that threshold and summaries from the 0.62 baseline, 200 comparisons each. The shape is the one you would expect from any precision-recall trade and the magnitudes are the point: the final 3.6 percentage points of name recall, from 0.956 to 0.992, quadruple the masked share of the corpus and cost 22 points of retrieval recall@5. That is not a tuning adjustment. It is a different product.

What makes the "99% everywhere" proposal wrong is the everywhere, since it applies one threshold to entity types whose costs differ by an order of magnitude. Person names are the expensive case because they are dense in a support corpus and they carry most of the discriminating signal. National identity numbers are the cheap case: they appear 4.1 times per 1,000 documents, they contribute nothing to retrieval, and nobody's query is improved by their presence. Running that recogniser at a threshold giving 0.995 recall and 0.40 precision costs the corpus roughly 0.02% of its tokens and no measurable retrieval quality. The same is true of card numbers, IBANs, and passport numbers.

So the answer to "push everything above 99%" is that you push the tier-1 entities well past it and you leave person names where the retrieval evidence says they belong, and you write the resulting per-entity thresholds into the same policy file that carries the tiers. There are situations where you pay the full cost anyway and accept the degraded product: a corpus destined for a provider endpoint whose terms allow training on inputs, a pipeline crossing into a jurisdiction where you have no lawful basis for the transfer, or a customer contract that names a specific standard. In those cases the honest move is to state that retrieval quality drops by roughly 22 points and to decide with that on the table, rather than to lower the threshold quietly and let the pilot team discover it.

Two cheap monitors would have caught the Tuesday change before the pilot team did. The first is the masked-token share of each ingestion batch, which moved from 1.4% to 7.9% in a single deploy and is a single counter divided by another; alerting on a relative jump above 50% costs nothing and fires on exactly this class of regression. The second is the distribution of placeholder types per batch, where a healthy corpus shows a long tail across a dozen entity types and the broken one showed 84% of all placeholders carrying a single label. Neither monitor requires a labelled corpus or a human, which matters because the recall measurement from chapter 3 is a quarterly exercise and ingestion changes weekly.

There is a tension in all of this that does not resolve. The property that makes a corpus useful for retrieval is that documents differ from one another in ways that track the entities they are about, and that is the same property that makes documents linkable to people. A surrogate scheme good enough to keep account-level retrieval working is, by construction, a scheme that preserves account-level linkage. You can move along that curve and you cannot leave it, which is why the tiering from chapter 2 does the real work: it decides which entities are allowed to remain distinguishable at all.

Everything up to here is a design and a set of measurements taken in a harness. None of it is evidence about the running system, where the detector might be invoked on one path and not another, at a threshold that drifted, over a payload the queue reconstructed from a retry. The next chapter is about producing that evidence.