I would not treat a low reconstruction score as proof that training data is safe. A sample can look normal to an auto-encoder yet teach your model the wrong behaviour. My starting point: check its source, label and effect on predictions - not just its score.
I would assess the filter in three settings: clean training data, poisoned data without filtering, and poisoned data after filtering. That shows whether screening reduces harm, rather than merely flags unusual records.
My checklist covers:
- Attack types: wrong labels, plausible labels and trigger-based backdoors.
- Attacker access: tests both without detector knowledge and with knowledge or feedback about the filter.
- Results: missed poisons, clean records retained, false alarms and targeted failures.
- Review: source records, label checks, subgroup results and separate post-training tests.
Lowering the threshold is not a cure. It can exclude valid records while letting poisons through. I would use documented review and temporary quarantine for suspect samples, then recheck thresholds when the data, supplier or model changes.
::: @figure 
{Poisoned Data Checks Beyond Reconstruction Error}
:::
Lecture 4: Training Data Poisoning
Why Poisoned Samples Can Have Low Scores
A low reconstruction error usually means a sample sits near the clean manifold. It does not mean the sample is safe.
Normal-Looking Features and Changed Labels
Clean-label and feature-collision attacks can keep samples close to the clean manifold. An auto-encoder may reconstruct them well even though they still cause harmful training behaviour.[5][8][10] Small feature changes can also preserve reconstruction while shifting classification. Check label agreement, nearby embeddings and targeted predictions together.[7][8]
Spread-Out Poisoning and Auto-Encoder Capacity
With enough records, an attacker can spread poison across many samples that each look plausible. Review per-sample scores, poison fraction and downstream harm, rather than relying on the mean alone.
An auto-encoder with enough capacity can also learn contaminated structure. Its behaviour depends on bottleneck size, regularisation and representation choice.[7][9] Testing must therefore account for the detector itself, rather than relying only on a threshold.
Overlap Between Poisons and Legitimate Variation
When benign data varies widely, clean and poisoned score distributions can overlap. Rare but valid records may score highly, while crafted poisons receive normal scores. Review scores by class and subgroup before lowering the threshold: doing so can remove useful data without reliably excluding poisons.
Testing Detector-Aware Attacks
Low-error samples can still cause harm. Detector-aware poisoning tests whether attackers can evade the filter itself by keeping samples below its threshold while preserving their downstream impact. “detector-aware” describes a threat model, not a single method.[11][14]
What Attackers Know and Control
Specify the attacker’s access: white-box, surrogate, black-box with repeated filtering feedback, or no detector knowledge. Record which inputs, labels, metadata and sample distributions they can change, and whether those changes affect the latent representation.
Also report the poisoning budget as both a record count and a percentage, alongside perturbation, realism and query limits. State whether detector training relies on trusted reference data or data the attacker can contaminate.[4][11]
Comparing Non-Adaptive and Detector-Aware Tests
Compare attacks with the same poisoning budget, downstream task and permitted changes. Document preprocessing, architectures, training schedules and random seeds. For surrogate attacks, explain how closely the substitute matches the deployed detector’s representation, loss and threshold.
| Test type | Attacker knowledge | Optimisation target | What the test checks | Key outputs |
|---|---|---|---|---|
| Non-adaptive | Ignores detector details | Downstream model harm | Whether ordinary poisons appear anomalous by chance | Detection metrics, poison budget and downstream attack impact |
| Surrogate-aware | Approximate detector or substitute model | Harm plus low surrogate score | Transferability across detector implementations | Surrogate-to-target transfer, score distributions and threshold results |
| Fully detector-aware | Target detector, representation and threshold | Harm subject to the filtering rule | Evasion of the deployed defence | Poison retention rate, detector scores, attack success and clean-data retention |
| Black-box adaptive | Only observed filtering or retraining outcomes | Harm while learning the defence response | Black-box or iterative evasion | Query count, evasion rate and variability across runs |
A failed non-adaptive attack does not prove resistance to adaptive attacks. Likewise, a failed adaptive attack covers only the access level, budget, architecture and data conditions tested.[3]
Measuring Detection and Downstream Harm
After defining the attack types, measure both detection and downstream harm at the chosen operating point. Report AUROC, precision, recall, false-positive rate, missed-poison count and retained clean data, with confidence intervals or ranges across runs.[13]
Compare attack success and clean-data performance across three settings: a clean baseline, an unfiltered poisoned model and the model retrained after filtering. Repeat the tests with different contamination levels, reference-data quality and threshold rules. Keep threshold tuning separate from evaluation.[4][12]
AUROC ranks samples; it does not show protection at the chosen operating point. Clean-test accuracy can also hide targeted or trigger-specific failures.[13]
Additional Checks and Limits of Filtering
Checks Before and After Training
Once a sample passes reconstruction screening, validate it further. Use reconstruction scores to guide investigation, not automatic deletion. Before ingestion, preserve source identities, permissions, timestamps and dataset versions. Then check schemas, labels, metadata and duplicates.[21][22]
Compare labels with embeddings, nearest neighbours and class prototypes. Check classification loss and whether predictions remain stable under benign transformations. Reserve gradients and influence scores for high-priority cases. Activation or spectral scans provide another signal, but may miss poisons that overlap clean representations.[16][19][20][23]
After training, test against a separate trusted hold-out, check subgroup errors and probe suspected backdoor triggers. If concerns remain, retrain controlled variants without the suspect records or source. Changes in behaviour call for further investigation. However, matching results across checks and benchmarks do not prove data integrity, especially when detectors share assumptions.[15][16][17][18]
Thresholds determine what the filter flags; post-training checks help reveal what it misses. Even strong detectors need thresholds for use in practice and human review.
Setting Thresholds and Reviewing Flagged Samples
Recheck score distributions whenever the supplier, model or data distribution changes. Compare flag rates across classes, languages, regions and relevant demographic groups. Rare records may look anomalous simply because they are poorly represented.
Start with read-only review. Then place suspicious samples in temporary quarantine, with documented expert review. Retain the original record, detector outputs, reviewer decision, date and affected model versions.
For each release, weigh the risk of missed poisons against reviewer hours, legitimate records excluded or restored, retraining duration, compute spend and time to redeploy. Assign separate responsibility for threshold approval, sample review and exclusion authorisation. Reserve expensive influence or ablation analyses for high-impact models and high-confidence cases.
Conclusion: Low Error Does Not Prove Data Is Clean
Across the tests above, low reconstruction error shows how well a sample fits the representation - not whether it is safe. Evaluation must therefore account for attackers.[7][9]
Test the filter against both non-adaptive poisoning and detector-aware attacks. Measure poison retention and downstream impact, not just detection rates.[25][26]
Use reconstruction as one signal alongside independent checks. Put borderline cases through controlled review rather than deleting them automatically. Reliability depends on reference-data quality, representation, model capacity, contamination and attack strategy. Thresholds can help bound risk, but they cannot prove that data is clean.[6][24][27]
FAQs
How can I spot poisoning when labels and reconstruction scores look normal?
Look beyond labels and reconstruction scores. Spectral signatures can flag unusual covariance patterns in a model’s internal feature space, even when inputs look clean [1]. Progressive perturbation checks how predictions change under small transformations that preserve the label, helping to flag suspicious samples [2].
Use these methods to prioritise samples for manual review, not as definitive proof. Legitimate rare data can trigger these filters too [1][2].
How can I test detector-aware attacks with limited resources?
Run a small, controlled pilot outside production with your own dataset and model family. Set compute and manual review budgets, along with acceptable false-positive rates. Score and filter the samples, then retrain to measure both attack success and accuracy on clean data. Include detector-aware variants that reduce separation between representations.
Refer uncertain cases for human review. Keep an audit trail to support rollbacks and refinements, and check published findings against your own data.
When should I quarantine data rather than adjust the filter?
Quarantine the data rather than adjust the filter if concerns about a sample’s label, provenance or behaviour remain unresolved [1]. Quarantine can be reversed and keeps the original sample, metadata and perturbation trace intact [1].
Before deciding whether to remove the suspect data, retrain models both with and without it. Compare performance, subgroup results and trigger behaviour. Permanent deletion requires strong, independent corroboration [1].