Research map / Adversarial robustness and security / Attacks on text and graphs
Reliable Adversarial Distillation with Unreliable Teachers
Jianing Zhu, Jiangchao Yao, Bo Han, Jingfeng Zhang, Tongliang Liu, Gang Niu, Jingren Zhou, Jianliang Xu, Hongxia Yang
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use IAD-I instead of ARD | adversarial distillation with ResNet-18 and TRADES teacher · CIFAR-10 | PGD-20 accuracy: 53.38% -> 53.84% | Improved |
| Use IAD-I instead of ARD | adversarial distillation with ResNet-18 and TRADES teacher · CIFAR-100 | PGD-20 accuracy: 30.21% -> 30.60% | Improved |
| Add student self-introspection to AKD² (ablation) | IAD-II, ResNet-18 · CIFAR-10 | PGD-20 accuracy: 51.36% -> 51.85% | Improved |
| Increase β from 0.01 to 1.0, raising student introspection (ablation) | IAD-I, ResNet-18 · CIFAR-10 | PGD-20 accuracy: 51.49% -> 52.64% | Improved |
| Increase β from 0.01 to 1.0, raising student introspection (ablation) | IAD-I, ResNet-18 · CIFAR-10 | Natural accuracy: 82.76% -> 80.79% | Hurt |
| Use 60-epoch rather than no warm-up (ablation) | IAD-I, ResNet-18 · CIFAR-10 | PGD-20 accuracy: 51.75% -> 52.14% | Improved |
| Use 60-epoch rather than no warm-up (ablation) | IAD-II, ResNet-18 · CIFAR-10 | PGD-20 accuracy: 52.07% -> 52.08% | No clear effect |
| Downweight teacher KL guidance by α instead of using constant weight 1.0 (ablation) | IAD-I-Down versus IAD-I · CIFAR-10 | PGD-20 accuracy: 52.14% -> 51.77% | Hurt |
| Downweight teacher KL guidance by α instead of using constant weight 1.0 (ablation) | IAD-I-Down versus IAD-I · CIFAR-100 | PGD-20 accuracy: 27.40% -> 27.32% | Hurt |
| Downweight teacher KL guidance by α instead of using constant weight 1.0 (ablation) | IAD-I-Down versus IAD-I · Tiny-ImageNet | PGD-20 accuracy: 23.40% -> 22.68% | Hurt |
| Increase the student-introspection coefficient from 1 - α to 4.0 (ablation) | IAD-I, ResNet-18 · CIFAR-10 | PGD-20 accuracy: 52.14% -> 55.78% | Improved |
| Increase the student-introspection coefficient from 1 - α to 4.0 (ablation) | IAD-I, ResNet-18 · CIFAR-10 | Natural accuracy: 82.09% -> 77.34% | Hurt |
Closest papers by meaning
- PeerAiD: Improving Adversarial Distillation from a Specialized Peer Tutor (CVPR 2024 · similarity 0.89)
- Learning Better Certified Models from Empirically-Robust Teachers (NeurIPS 2026 · similarity 0.89)
- Improving Adversarial Robust Fairness via Anti-Bias Soft Label Distillation (NeurIPS 2024 · similarity 0.88)
- Annealing Self-Distillation Rectification Improves Adversarial Training (ICLR 2024 · similarity 0.87)
- Impartial Adversarial Distillation: Addressing Biased Data-Free Knowledge Distillation via Adaptive Constrained Optimization (AAAI 2024 · similarity 0.87)
- Robust Distillation via Untargeted and Targeted Intermediate Adversarial Samples (CVPR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
