Research map / Adversarial robustness and security / Certified and empirical robustness
Enhanced Accuracy and Robustness via Multi-Teacher Adversarial Distillation
Shiji Zhao, Jie Yu, Zhenlong Sun, Bo Zhang, Xingxing Wei
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add Adaptive Normalization Loss to dynamically weight clean and adversarial teachers (Baseline+MT+ANL) (ablation) | Baseline+MT · CIFAR-10 ablation (Figure 2) | W-Robust accuracy | Improved |
| add a clean teacher alongside the adversarial teacher with fixed 0.5 weights (Baseline+MT) (ablation) | Baseline · CIFAR-10 ablation (Figure 2) | clean and robust accuracy | Mixed |
| add Adaptive Normalization Loss: training loss less oscillating and relative loss gap smaller (ablation) | Baseline+MT · CIFAR-10 training loss (Figures 3, 4) | training loss | Improved |
| add clean teacher to adversarial distillation (MTARD vs RSLAD), clean accuracy | RSLAD · CIFAR-10 | Clean accuracy: 83.99% -> 87.36% | Improved |
| add clean teacher to adversarial distillation (MTARD vs RSLAD), clean accuracy | RSLAD · CIFAR-100 | Clean accuracy: 58.25% -> 64.3% | Improved |
| add clean teacher with adaptive weights (MTARD vs RSLAD), PGDsat robust accuracy | RSLAD · CIFAR-10 white-box PGDsat | Robust accuracy: 53.94% -> 50.73% | Hurt |
| MTARD vs RSLAD under FGSM W-Robust accuracy | RSLAD · CIFAR-10 white-box FGSM | W-Robust accuracy: 72.2% -> 74.28% | Improved |
| MTARD vs RSLAD under PGDsat W-Robust accuracy | RSLAD · CIFAR-100 white-box PGDsat | W-Robust accuracy: 44.72% -> 44.63% | No clear effect |
| MTARD vs RSLAD under CW-infinity W-Robust accuracy | RSLAD · CIFAR-10 white-box CW | W-Robust accuracy: 68.33% -> 67.97% | Hurt |
| MTARD vs ARD under Square attack W-Robust accuracy | ARD · CIFAR-10 black-box Square attack | W-Robust accuracy: 78.71% -> 83.68% | Improved |
| MTARD vs ARD under PGD-20 transfer attack W-Robust accuracy | ARD · CIFAR-100 black-box PGD-20 transfer | W-Robust accuracy: 49.82% -> 52.85% | Improved |
| use adversarial distillation (ARD, RSLAD, MTARD) instead of SAT and TRADES | small student models · CIFAR-10/100 white-box | W-Robust accuracy | Improved |
Closest papers by meaning
- Mitigating Accuracy-Robustness Trade-Off via Balanced Multi-Teacher Adversarial Distillation (TPAMI 2024 · similarity 0.94)
- Learning Better Certified Models from Empirically-Robust Teachers (NeurIPS 2026 · similarity 0.90)
- Out of Thin Air: Exploring Data-Free Adversarial Robustness Distillation (AAAI 2024 · similarity 0.89)
- Indirect Gradient Matching for Adversarial Robust Distillation (ICLR 2025 · similarity 0.89)
- Robust Distillation via Untargeted and Targeted Intermediate Adversarial Samples (CVPR 2024 · similarity 0.88)
- PeerAiD: Improving Adversarial Distillation from a Specialized Peer Tutor (CVPR 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
