Research map / Adversarial robustness and security / Certified and empirical robustness
Adversarially Robust Distillation
Micah Goldblum, Liam Fowl, Soheil Feizi, Tom Goldstein
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use ARD instead of clean-image distillation from a robust ResNet18 teacher | MobileNetV2 · CIFAR-10 | robust accuracy (20-step PGD): 38.21% -> 50.22% | Improved |
| Distill from an adversarially trained teacher using clean images | ResNet18 and MobileNetV2 students · CIFAR-10 | robust accuracy (20-step PGD): 0.0% -> 40.13%; 0.0% -> 38.21% | Improved |
| Use clean-image distillation from a robust WRN teacher rather than ARD | MobileNetV2 · CIFAR-10 | robust accuracy (20-step PGD): 5.46%; 21.69% | Hurt |
| Use ARD from a TRADES WRN teacher instead of adversarial training | MobileNetV2 · CIFAR-10 | robust accuracy (20-step PGD): 46.90% -> 50.42% | Improved |
| Set α=0.95 instead of α=1 (ablation) | TRADES WRN ARD → MobileNetV2 · CIFAR-10 | natural accuracy: 82.63% -> 84.70% | Improved |
| Set α=0.95 instead of α=1 (ablation) | TRADES WRN ARD → MobileNetV2 · CIFAR-10 | robust accuracy (20-step PGD): 50.42% -> 46.28% | Hurt |
| Use ARD from a robust WRN teacher instead of adversarial training | MobileNetV2 · CIFAR-100 | robust accuracy (20-step PGD): 22.80% -> 27.64% | Improved |
| Use ARD distillation from ResNet18 instead of adversarial training | ResNet18 · CIFAR-10 | robust accuracy (20-step PGD): 44.46% -> 51.21% | Improved |
| Use ARD distillation from MobileNetV2 instead of adversarial training | MobileNetV2 · CIFAR-10 | robust accuracy (20-step PGD): 46.90% -> 47.95% | Improved |
| Use Fast-ARD rather than free adversarial training | MobileNetV2 (m=4) · CIFAR-10 | robust accuracy (20-step PGD): 23.13% -> 37.07% | Improved |
| Replace full ARD with Fast-ARD | MobileNetV2 · CIFAR-10 | training time: 41.06 -> 5.15 hrs | Improved |
| Replace adversarial training with ARD | MobileNetV2 · CIFAR-10 under MI-FGSM20 | robust validation accuracy: 50.82% -> 55.16% | Improved |
Closest papers by meaning
- Learning Better Certified Models from Empirically-Robust Teachers (NeurIPS 2026 · similarity 0.93)
- Out of Thin Air: Exploring Data-Free Adversarial Robustness Distillation (AAAI 2024 · similarity 0.93)
- Robust Distillation via Untargeted and Targeted Intermediate Adversarial Samples (CVPR 2024 · similarity 0.92)
- Improving Adversarial Robust Fairness via Anti-Bias Soft Label Distillation (NeurIPS 2024 · similarity 0.91)
- PeerAiD: Improving Adversarial Distillation from a Specialized Peer Tutor (CVPR 2024 · similarity 0.90)
- Towards Adversarially Robust Dataset Distillation by Curvature Regularization (AAAI 2025 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
