Research map / Adversarial robustness and security / Backdoor and poisoning
From Toxic to Trustworthy: Using Self-Distillation and Semi-supervised Methods to Refine Neural Networks
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes FTT, a two-stage backdoor-removal method combining self-attention distillation on a small clean set with semi-supervised learning (FixMatch-style pseudo-labeling) on poisoned data with labels discarded, to erase backdoors while improving accuracy.
Problem
Backdoor attacks implant triggers in DNNs; existing backdoor erasure methods fail against strong attacks (e.g., ATTEQ-NN) and typically reduce clean accuracy, especially when only a small clean dataset is available.
Approach
Self-distillation via early-exit branches with attention-map alignment to the deepest network removes backdoors using 5% clean data, then SSL treats poisoned training data as unlabeled (labels discarded, pseudo-labeled by high-confidence weakly augmented predictions) to recover accuracy.
Key results
- FTT(SAD+SSL) reduces average ASR on CIFAR-10 PreActResNet-18 from 97.03% to 1.89% and raises average accuracy by 4.43% (to 95.20%) across five attacks.
- FTT(SAD) alone reduces average ASR to 4.49%, versus 60.62% (finetuning), 26.10% (fine-pruning), 49.49% (NAD), and 18.42% (ANP).
- FTT is the only evaluated defense that reduces ASR below 10% with only 0.5% clean data, and it reduces ATTEQ-NN ASR to 6.74% where other methods fail.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add SSL after SAD (ablation) | FTT on CIFAR-10 with BadNets · CIFAR-10, BadNets | ASR: 0.82 -> 0.61 | Helped |
| Add SSL after SAD (ablation) | FTT on CIFAR-10 with BadNets · CIFAR-10, BadNets | ACC: 91.53 -> 95.34 | Helped |
| Add SSL after SAD (ablation) | FTT on CIFAR-10 with Blended · CIFAR-10, Blended | ASR: 8.42 -> 0.93 | Helped |
| Add SSL after SAD (ablation) | FTT on CIFAR-10 with Blended · CIFAR-10, Blended | ACC: 91.57 -> 95.01 | Helped |
| Add SSL after SAD (ablation) | FTT on CIFAR-10 with ATTEQ-NN · CIFAR-10, ATTEQ-NN | ASR: 8.41 -> 6.06 | Helped |
| Add SSL after SAD (ablation) | FTT on CIFAR-10 with ATTEQ-NN · CIFAR-10, ATTEQ-NN | ACC: 85.35 -> 93.02 | Helped |
| Add SAD to SSL (ablation) | FTT on CIFAR-10 with BadNets · CIFAR-10, BadNets | ASR: 23.33 -> 0.61 | Helped |
| Add SAD to SSL (ablation) | FTT on CIFAR-10 with BadNets · CIFAR-10, BadNets | ACC: 63.28 -> 95.34 | Helped |
| Add SAD to SSL (ablation) | FTT on CIFAR-10 with Blended · CIFAR-10, Blended | ASR: 99.93 -> 0.93 | Helped |
| Add SAD to SSL (ablation) | FTT on CIFAR-10 with Blended · CIFAR-10, Blended | ACC: 91.99 -> 95.01 | Helped |
| Add SAD to SSL (ablation) | FTT on CIFAR-10 with ATTEQ-NN · CIFAR-10, ATTEQ-NN | ASR: 99.36 -> 6.06 | Helped |
| Add SAD to SSL (ablation) | FTT on CIFAR-10 with ATTEQ-NN · CIFAR-10, ATTEQ-NN | ACC: 70.15 -> 93.02 | Helped |
Surprising findings
- Poisoning ratio of unlabeled data (0% to 75%) has little effect on SSL results after self-distillation purification.
- Baseline defenses such as ANP reduced clean accuracy by about 10 points on CIFAR-10 while lowering ASR.
Limitations
- Relies on access to a small clean dataset (about 5% of training data) for the SAD stage.
- Evaluated only on CIFAR-10/100 with ResNet-style architectures and six attacks.
- Some paper claims are loosely supported; for example, the claimed 2x ASR reduction and 5% accuracy gain versus SOTA are relative to a selected set of baselines.
Experimental setup
Datasets: CIFAR-10 CIFAR-100
Benchmarks: CIFAR-10 CIFAR-100
Metrics: attack success rate accuracy
Baselines: Finetuning Fine-pruning NAD ANP
Builds on: Neural Attention Distillation FixMatch Be Your Own Teacher self-distillation attention transfer
Closest papers by meaning
- Uncovering, Explaining, and Mitigating the Superficial Safety of Backdoor Defense (NeurIPS 2024 · similarity 0.91)
- Unveiling and Mitigating Backdoor Vulnerabilities based on Unlearning Weight Changes and Backdoor Activeness (NeurIPS 2024 · similarity 0.90)
- BackdoorDM: A Comprehensive Benchmark for Backdoor Learning on Diffusion Model (NeurIPS 2025 · similarity 0.90)
- Rethinking Backdoor Attacks on Dataset Distillation: A Kernel Method Perspective (ICLR 2024 · similarity 0.89)
- BaDExpert: Extracting Backdoor Functionality for Accurate Backdoor Input Detection (ICLR 2024 · similarity 0.88)
- REFINE: Inversion-Free Backdoor Defense via Model Reprogramming (ICLR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.