Research map / Adversarial robustness and security / Certified and empirical robustness
Adversarially Robust Distillation by Reducing the Student-Teacher Variance Gap
Junhao Dong, Piotr Koniusz, Junxi Chen, Yew-Soon Ong
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add covariance and gram-matrix alignment components to the distillation loss (FCA, PGM, Omega) (ablation) | ARKD baseline · CIFAR-10 (WRN-28 to ResNet-18) | AutoAttack robust accuracy: 51.27 -> 53.78 | Improved |
| use STARSHIP instead of RSLAD for WRN-28 to ResNet-18 distillation | robust knowledge distillation · CIFAR-10 | AutoAttack robust accuracy: 51.36 -> 53.78 | Improved |
| add statistical sub-matrices alignment (Omega) (ablation) | STARSHIP · CIFAR-10 | clean accuracy: 85.35 -> 86.47 | Improved |
| use MaxExp distance for covariance alignment instead of Frobenius norm (ablation) | STARSHIP · CIFAR-10 | AutoAttack robust accuracy: 52.10 -> 53.78 | Improved |
| use adaptive adversary generation (Ada-STARSHIP) instead of standard | STARSHIP · CIFAR-100 (WRN-28 to ResNet-18) | AutoAttack robust accuracy: clean 61.54 -> 62.19; AA 29.30 -> 28.28 | Mixed |
| increase beta to weight adversarial alignment more (ablation) | STARSHIP · CIFAR-10 | clean vs robust accuracy | Mixed |
| add 1M DDPM synthetic training data (ablation) | STARSHIP · CIFAR-10 (WRN-28 to ResNet-18) | AutoAttack robust accuracy: 53.78 -> 54.49 | Improved |
| use single-step N-FGSM adversary instead of multi-step (ablation) | STARSHIP · CIFAR-10 (WRN-28 to ResNet-18) | robust accuracy and time: robust 51.06; 50 s per epoch vs AdaAD 112 s | Mixed |
| distill from ViT-B teacher with STARSHIP | ResNet-18 student · CIFAR-10 | AutoAttack robust accuracy: 49.66 (teacher) -> 52.07 | Improved |
| use STARSHIP in self-distillation on CIFAR-10 ResNet-18 | TRADES teacher ResNet-18 · CIFAR-10 self-distillation | AutoAttack robust accuracy: 48.90 -> 52.42 | Improved |
Closest papers by meaning
- Toward Understanding Adversarial Distillation: Why Robust Teachers Fail (ICML 2026 · similarity 0.94)
- Allies Teach Better Than Enemies: Inverse Adversaries for Robust Knowledge Distillation (TPAMI 2026 · similarity 0.94)
- Robust Distillation via Untargeted and Targeted Intermediate Adversarial Samples (CVPR 2024 · similarity 0.93)
- Learning Better Certified Models from Empirically-Robust Teachers (NeurIPS 2026 · similarity 0.91)
- Towards Adversarially Robust Dataset Distillation by Curvature Regularization (AAAI 2025 · similarity 0.89)
- Improving Adversarial Robust Fairness via Anti-Bias Soft Label Distillation (NeurIPS 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
