How well does Tenet detect PII/PHI?

Detection accuracy is measured against all 18 HIPAA Safe Harbor identifier categories using a synthetic annotated benchmark corpus. Data below are from the full Tenet pipeline: piiranha ML + regex augmentation + Presidio fallback + clinical NER.

0.985 Avg AUC across 6 HIPAA-critical types
15 of 18 HIPAA Safe Harbor identifier categories covered
6 languages EN · ES · FR · DE · IT · NL
2,710 samples annotated across 15 categories, 500 adversarial negatives
3 detection layers ML + regex + NER fallback

All benchmarks use synthetic data. Real-world performance may vary. Biometric identifiers and full-face photographs are not covered.

Detection accuracy by HIPAA identifier category

AUC scores from ROC analysis across the full detection pipeline

HIPAA Category AUC Method
Dates (DOB) 1.000
ML + regex
SSN 0.999
ML
Email 0.994
ML
Account Numbers 0.991
ML
Names (Given + Surname) 0.975
ML + Presidio
Phone/Fax 0.949
ML
Geographic (Street, City, ZIP) 0.932
ML
Other Unique IDs (Tax, CC) 0.885
ML + regex
License Numbers 0.650
ML
Device Identifiers 0.642
ML
IP Addresses
Regex-only
VIN
Regex-only
Medical Record #
Regex-only
Web URLs
Regex-only
Biometric
Not covered

8 of 10 ML-scored categories exceed AUC 0.88. The 6 HIPAA-critical types average 0.98 AUC.

Precision-recall tradeoffs at optimal thresholds

Configured thresholds balance recall (catching PHI) against false positive rate (flagging clean data)

Category Threshold Recall (TPR) False Positive Rate
Dates 0.94 100% 0.0%
SSN 0.97 100% 0.4%
Email 0.66 98.8% 0.0%
Account Numbers 0.82 98.1% 0.0%
Names 0.45 96.0% 1.0%
Phone/Fax 0.44 90.5% 0.4%
Geographic 0.51 86.7% 0.8%

Lower thresholds on Names and Geographic reflect inherent ambiguity — these categories have higher irreducible false positive rates than structured identifiers.

Clinical entity detection

Separate model evaluated on diagnosis, treatment, and lab test extraction from clinical notes

Clinical Entity AUC Positives Negatives
Diagnosis 0.906 48 122
Treatment 0.903 41 129
Lab Test 0.745 14 156

Lab test AUC reflects limited positive sample size (n=14) — interpret with caution. Diagnosis and treatment detection are production-ready.

ROC curves

Full receiver operating characteristic curves for each HIPAA identifier category and clinical entity type

ROC curve: All HIPAA categories (overview)
ROC curve: All HIPAA categories (overview)

All HIPAA Categories

ROC curve: Names
ROC curve: Names

Names AUC 0.975

ROC curve: Dates / DOB
ROC curve: Dates / DOB

Dates / DOB AUC 1.000

ROC curve: Email
ROC curve: Email

Email AUC 0.994

ROC curve: SSN
ROC curve: SSN

SSN AUC 0.999

ROC curve: Phone / Fax
ROC curve: Phone / Fax

Phone / Fax AUC 0.949

ROC curve: Geographic identifiers
ROC curve: Geographic identifiers

Geographic AUC 0.932

ROC curve: Account numbers
ROC curve: Account numbers

Account Numbers AUC 0.991

ROC curve: Clinical NER (overview)
ROC curve: Clinical NER (overview)

All Clinical Entities

ROC curve: Diagnosis
ROC curve: Diagnosis

Diagnosis AUC 0.906

ROC curve: Treatment
ROC curve: Treatment

Treatment AUC 0.903

ROC curve: Lab test
ROC curve: Lab test

Lab Test AUC 0.745

Enforced benchmark quality floors

The benchmark suite fails CI if any of these thresholds are not met

Per-type recall floor

70% minimum — HIPAA-critical types are hard failures

False positive rate ceiling

15% maximum including adversarial samples

Micro-F1 floor

60% minimum across all entity types

Safe Harbor coverage

Minimum 8 of 18 HIPAA identifier categories