Research map / Adversarial robustness and security / Backdoor and poisoning
Attack To Defend: Exploiting Adversarial Attacks for Detecting Poisoned Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes A2D, a black-box poisoned-model detector that crafts strong PGD adversarial examples on a reference model trained on a small clean set and transfers them to the target model, flagging it as trojan if its adversarial sensitivity (SAP) is high.
Problem
Detecting whether a given (black-box) ML model has been backdoored/poisoned by a trigger-based poisoning attack, in a way that generalizes to unseen and stealthy attacks without shadow models, full training data, or white-box access.
Approach
Poisoned models are more sensitive to adversarial perturbations than benign ones; the defender generates strong adversarial examples against an unrelated reference model trained on a small clean set, transfers them to the target, and uses a relative accuracy-drop metric (sensitivity to adversarial perturbations, SAP) as the trojan score.
Key results
- A2D achieves 100% detection accuracy on Modify, WaNet, and IAD attacks on MNIST and high accuracy across attacks on CIFAR10 and GTSRB, outperforming MNTD, ULP, NC, FRE and CAS on most attacks.
- On chest X-ray, A2D reaches 80% (Blend), 87.5% (WaNet), and 100% (IAD) accuracy versus 75%, 50%, 50% for ET.
- Reference model architecture has little effect on detection accuracy; black-box (different architecture) detection performs comparably to gray-box.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use A2D instead of the strongest listed baseline | Poisoned-model detection · MNIST Modify | ACC (%): 94 -> 94.6 | Helped |
| Use A2D instead of the strongest listed baseline | Poisoned-model detection · MNIST Blend | ACC (%): 77 -> 99.6 | Helped |
| Use A2D instead of the strongest listed baseline | Poisoned-model detection · MNIST WaNet | ACC (%): 87.5 -> 100 | Helped |
| Use A2D instead of the strongest listed baseline | Poisoned-model detection · MNIST IAD | ACC (%): 82.5 -> 100 | Helped |
| Use A2D instead of the strongest listed baseline | Poisoned-model detection · CIFAR10 Modify | ACC (%): 91 -> 93 | Helped |
| Use A2D instead of the strongest listed baseline | Poisoned-model detection · CIFAR10 Blend | ACC (%): 85 -> 86.6 | Helped |
| Use A2D instead of the strongest listed baseline | Poisoned-model detection · CIFAR10 WaNet | ACC (%): 75 -> 100 | Helped |
| Use A2D instead of the strongest listed baseline | Poisoned-model detection · CIFAR10 IAD | ACC (%): 80 -> 95 | Helped |
| Use A2D instead of the strongest listed baseline | Poisoned-model detection · CIFAR10 LC | ACC (%): 85 -> 95 | Helped |
| Use A2D instead of the strongest listed baseline | Poisoned-model detection · CIFAR10 SIG | ACC (%): 90 -> 100 | Helped |
| Use A2D instead of the strongest listed baseline | Poisoned-model detection · CIFAR10 ISS | ACC (%): 62.5 -> 100 | Helped |
| Use A2D instead of the strongest listed baseline | Poisoned-model detection · GTSRB WaNet | ACC (%): 92.5 -> 100 | Helped |
Surprising findings
- Adaptive attacks that add an adversarial-robustness loss to poisoning cause a substantial ASR drop, showing a trade-off between evading A2D and keeping the backdoor effective.
- Poisoned models show higher IOU between backdoor and adversarial activation pathways than benign models.
Limitations
- Requires a small clean set from the defender.
- Margin parameter must be small (omega <= 0.1) and at least about 100 attack samples are needed for reliable detection.
- Black-box transfer generalization to vision transformers is not verified.
Experimental setup
Datasets: MNIST CIFAR10 GTSRB Chest X-ray
Benchmarks: MNIST CIFAR10 GTSRB Chest X-ray
Metrics: ACC TDR FDR AUC SAP
Baselines: MNTD ULP CAS Neural Cleanse FeatureRE ET (expected transferability)
Builds on: Cassandra (CAS) PGD adversarial attack Neural Cleanse
Closest papers by meaning
- Witches' Brew: Industrial Scale Data Poisoning via Gradient Matching (ICLR 2021 · similarity 0.89)
- Adversarial-Inspired Backdoor Defense via Bridging Backdoor and Adversarial Attacks (AAAI 2025 · similarity 0.87)
- Adversarial Training for Defense Against Label Poisoning Attacks (ICLR 2025 · similarity 0.87)
- On the Adversarial Risk of Test Time Adaptation: An Investigation into Realistic Test-Time Data Poisoning (ICLR 2025 · similarity 0.87)
- Sharpness-Aware Data Poisoning Attack (ICLR 2024 · similarity 0.87)
- Universal Backdoor Attacks (ICLR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.