Research map / Adversarial robustness and security / Backdoor and poisoning
Neural Attention Distillation: Erasing Backdoor Triggers from Deep Neural Networks
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| distill intermediate attention from a finetuned teacher into the backdoored student | WRN-16-1 · CIFAR-10/GTSRB, 6-attack average | ASR: 99.04 -> 7.22 | Helped |
| distill intermediate attention from a finetuned teacher into the backdoored student | WRN-16-1 · CIFAR-10/GTSRB, 6-attack average | ACC: 83.50 -> 80.83 | Hurt |
| replace attention-map distillation with raw activation-map distillation (ablation) | NAD · CIFAR-10 BadNets | ASR: 3.81% -> 98.44% | Hurt |
| use squared-sum attention instead of mean or sum attention (ablation) | NAD · CIFAR-10 BadNets, Epoch 5 | ASR: 12.28% / 4.60% / 6.89% -> 4.21% | Helped |
| repeat attention distillation for multiple iterations rather than distilling once (ablation) | NAD · CIFAR-10 BadNets | ASR: increased by 2% | Hurt |
| use a trained-from-scratch clean-data teacher rather than the finetuned backdoored teacher (ablation) | NAD with backdoored student · CIFAR-10, teacher-student combinations | ASR: more than 80% | Helped |
| use a trained-from-scratch teacher trained on only 5% clean data (ablation) | NAD with backdoored student · CIFAR-10, teacher-student combinations | ACC | Hurt |
| use the clean-finetuned backdoored network as both teacher and student (ablation) | NAD · CIFAR-10, teacher-student combinations | ASR | Helped |
| use different MCR endpoint models rather than two copies of the backdoored model (ablation) | MCR · CIFAR-10, six attacks | ASR: BadNets 4.65% -> 6.00%; Trojan 41.25% -> 53.31%; Blend 64.33% -> 70.65%; CL 32.95% -> 42.66%; SIG 1.62% -> 7.32%; Refool | Hurt |
| use NAD rather than finetuning alone | WRN-16-1 · CIFAR-10/GTSRB, 6-attack average | ASR: 37.36 -> 7.22 | Helped |
| use NAD rather than MCR | WRN-16-1 · CIFAR-10/GTSRB, 6-attack average | ASR: 25.59 -> 7.22 | Helped |
| use NAD rather than Fine-pruning | WRN-16-1 · CIFAR-10/GTSRB, 6-attack average | ASR: 67.00 -> 7.22 | Helped |
Closest papers by meaning
- From Toxic to Trustworthy: Using Self-Distillation and Semi-supervised Methods to Refine Neural Networks (AAAI 2024 · similarity 0.90)
- Unveiling and Mitigating Backdoor Vulnerabilities based on Unlearning Weight Changes and Backdoor Activeness (NeurIPS 2024 · similarity 0.89)
- Taught Well Learned Ill: Towards Distillation-conditional Backdoor Attack (NeurIPS 2025 · similarity 0.88)
- Rethinking Backdoor Attacks on Dataset Distillation: A Kernel Method Perspective (ICLR 2024 · similarity 0.88)
- REFINE: Inversion-Free Backdoor Defense via Model Reprogramming (ICLR 2025 · similarity 0.88)
- Uncovering, Explaining, and Mitigating the Superficial Safety of Backdoor Defense (NeurIPS 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.