Research map / Adversarial robustness and security / Certified and empirical robustness
Mitigating Accuracy-Robustness Trade-Off via Balanced Multi-Teacher Adversarial Distillation
Shiji Zhao, Xizhe Wang, Xingxing Wei
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add Normalization Loss Balance to baseline multi-teacher distillation (ablation) | Baseline · CIFAR-10 PGD-sat | W-Robust Acc: +0.87% | Improved |
| add Normalization Loss Balance to baseline multi-teacher distillation (ablation) | Baseline · CIFAR-10 PGD-trades | W-Robust Acc: +0.56% | Improved |
| add Normalization Loss Balance to baseline multi-teacher distillation (ablation) | Baseline · CIFAR-10 CW | W-Robust Acc: +0.6% | Improved |
| add Entropy-Based Balance on top of Baseline+NLB (ablation) | MTARD (Baseline+NLB) · CIFAR-10 PGD-sat | W-Robust Acc: +0.89% | Improved |
| add Entropy-Based Balance on top of Baseline+NLB (ablation) | MTARD (Baseline+NLB) · CIFAR-10 PGD-trades | W-Robust Acc: +0.82% | Improved |
| add Entropy-Based Balance on top of Baseline+NLB (ablation) | MTARD (Baseline+NLB) · CIFAR-10 CW | W-Robust Acc: +0.94% | Improved |
| add Normalization Loss Balance to Baseline+EBB (ablation) | Baseline+EBB · CIFAR-10 PGD-sat | W-Robust Acc: +0.33% | Improved |
| entropy-based balance of teacher knowledge scale (ablation) | B-MTARD · CIFAR-10 teacher knowledge scale | knowledge scale ResNet-56: 2.2518 -> 1.2480 | Improved |
| raise initial temperature tau from 1 to 8 (ablation) | B-MTARD · CIFAR-10 PGD-sat | W-Robust Acc: 69.94% -> 67.81% | Hurt |
| raise beta from 1 to 10 (ablation) | B-MTARD · CIFAR-10 PGD-sat | W-Robust Acc: 69.94% -> 68.86% | Hurt |
| lower beta from 1 to 0.5 (ablation) | B-MTARD · CIFAR-10 PGD-sat | W-Robust Acc: 69.94% -> 69.38% | Hurt |
| replace single/robust-only distillation with balanced multi-teacher (MTARD without balance) on clean accuracy | ARD/RSLAD · CIFAR-10 | clean accuracy: 83.99% -> 88.20% | Improved |
Closest papers by meaning
- Enhanced Accuracy and Robustness via Multi-Teacher Adversarial Distillation (ECCV 2022 · similarity 0.94)
- Adversarially Robust Distillation by Reducing the Student-Teacher Variance Gap (ECCV 2024 · similarity 0.91)
- Learning Better Certified Models from Empirically-Robust Teachers (NeurIPS 2026 · similarity 0.90)
- Robust Distillation via Untargeted and Targeted Intermediate Adversarial Samples (CVPR 2024 · similarity 0.90)
- Out of Thin Air: Exploring Data-Free Adversarial Robustness Distillation (AAAI 2024 · similarity 0.89)
- Improving Adversarial Robust Fairness via Anti-Bias Soft Label Distillation (NeurIPS 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
