Module 2 · Running ingest in production · Lesson 7
PII detection and redaction
Where personal data flows in an ingest pipeline, which uses need it and which don't, how to detect it with rules, NER and LLMs, how to pseudonymise consistently and redact PDFs properly, and how to measure recall under data-protection law such as GDPR.
By the end of this lesson you can
- Map where personal data flows through ingest and decide, use by use, whether it needs identified, pseudonymised or redacted data under data minimisation
- Distinguish redaction, masking, pseudonymisation and anonymisation as the ICO uses the terms, and explain why title numbers and addresses are quasi-identifiers
- Implement detection with Presidio's pattern recognisers plus custom domain recognisers, consistent keyed pseudonyms, and an LLM second pass with span verification
- Redact PDFs so the underlying text is removed, and measure detection recall and the leaks it implies per case