PII detection and redaction in AI pipelines

Read these first

• Versioning and lineage for evaluation sets

What you will know

• Detect and redact PII before it reaches a model or a log

  Cover structured fields, free text, traces, and error payloads without assuming that application-level filtering catches every copy.

• Measure the false-negative rate of the detector

  Use a reviewed, representative corpus stratified by entity type and language; false positives matter too because excessive masking removes useful context.

• Instrument pipeline boundaries to prove redaction precedes provider and observability sinks

  Retries, queues, traces, caches, and dead-letter streams can receive the original payload even when the main request path is clean.

• Handle PII already in an index

  Removal usually requires identifying affected records, deleting or reindexing them, and checking replicas, caches, and backups for residual copies.

• Trace a sample payload through providers, logs, and retention systems and record what cannot be recalled

  External processors may retain prompts, abuse-review copies, or derived artifacts beyond your deletion control, so the send decision must account for that boundary.

Chapters

1. 1

   A detector that was never invoked is not a detection failure

   Two incidents land in one week: a phone number in a local format the detector does not recognise, and a customer's full name in your traces because the tracing middleware records the request span before the handler redacts. Only the first is about detection — a perfect detector changes nothing about the second. Redaction placed at a single call site leaves every other exit open, and trace spans, serialised errors, retries, queues, caches and the dead-letter stream all carry the original payload. Structured fields are the easy half and a different problem: a typed column is known in advance, so it is never selected into the payload at all, or is swapped for a stable token at the serialisation layer, with no detector and no recall question.

   Read chapter
2. 2

   Entity types differ in harm, and some are personal only together

   Stratifying a recall measurement needs an axis, and converting two recall numbers into a decision needs a scale. A national identifier or a card number is not the disclosure a work email address is, so 71% on the first can be a stop while 96% on the second is a ship — once each is weighed against how often that entity actually appears in the traffic you are sending. Build the ranking before you measure anything. Then note what an entity-by-entity view structurally cannot see: a postcode, a birth date and a job title re-identify a person jointly while no single field is personal data on its own, and whether a string is personal at all depends on the context it sits in and the jurisdiction you are in.

   Read chapter
3. 3

   Precision says nothing about what the detector let through

   A vendor advertises 99.1% accuracy and your own test set reports 99.2% precision, and neither is the number a sign-off rests on. Recall is: a detector that flags one email and gets it right scores perfectly on precision while missing everything else. Produce per-entity recall on a reviewed, representative sample of your own traffic, stratified by entity type and language — 96% on emails beside 71% on national ID numbers is a decision to make, not a footnote.

   Read chapter
4. 4

   Masking every flag strips the tokens that distinguish one chunk

   Mask every detected person name before embedding a support corpus and retrieval on account-specific questions collapses, with chunks from unrelated tickets returned interchangeably for almost any account query. The detector was also flagging product names, company names and error codes, and a single placeholder replaced them all, leaving chunks that no longer differ. The constructive form is typed, per-entity-stable surrogates that preserve distinguishability and coreference within a document. Here you also price the other direction: what forcing recall above 99% on every entity type costs in destroyed context, and when to pay it anyway.

   Read chapter
5. 5

   A canary found in a sink is evidence; a unit test is a claim

   Unit tests covering every entity type show that the redactor works when it is called, which is not the assertion being made. Drive a payload carrying a unique marker through the pipeline, then search for that marker in every sink, failure paths included. You will build the CI version and the continuously running production version, obtain a realistic payload without creating the exposure you are testing for, and see why running the detector at the outbound client is circular reasoning.

   Read chapter
6. 6

   Sending to a provider is irrevocable, so the send is the control

   Once a transcript reaches an external model provider, retention is governed by their terms, and abuse-review copies and derived artefacts can outlive any deletion request you file. That makes the decision to send the real control point rather than anything downstream of it. Trace one sample payload through providers, logs and retention systems, and write down which parts of its journey you can recall and which you cannot — the second list is what a sign-off is actually about.

   Read chapter
7. 7

   Removing PII from an index means finding every copy of the chunk

   A transcript holding a customer's full name, home address and partial card number was ingested four months ago and has been retrieved into provider prompts an unknown number of times. Removal covers the vectors and the stored chunk text, every read replica and cache, every backup taken since, and derived artefacts such as eval sets and cached retrieval results. This chapter orders the remediation and works out whether content-derived chunk ids in your retrieval logs help you here or expose you further.

   Read chapter