Research map / Adversarial robustness and security / Certified and empirical robustness
PeerAiD: Improving Adversarial Distillation from a Specialized Peer Tutor
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes PeerAiD, an online adversarial distillation method where a peer network trained alongside the student learns to defend against student-generated adversarial examples, improving student robustness on CIFAR and TinyImageNet.
Problem
Adversarial distillation uses a fixed pretrained robust teacher whose robustness degrades against adversarial examples generated from the student during distillation, limiting the guidance quality for small student networks.
Approach
Train a peer network from scratch simultaneously with the student, using adversarial examples crafted against the student, so the peer specializes in defending the student (not itself); the peer's soft labels guide both inner maximization and outer minimization.
Key results
- PeerAiD improves AutoAttack accuracy of ResNet-18 over all baselines by 0.73 to 1.66 points on the reported benchmarks.
- PeerAiD's student surpasses clean accuracy of other adversarial distillation baselines by up to 4.72 points with ResNet-18 on TinyImageNet.
- The peer model reaches 75.63% natural accuracy versus 57.30% for the pretrained robust teacher on CIFAR-100, while its PGD-20 robust accuracy against itself is 0%.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use PeerAiD instead of adversarial distillation baselines | ResNet-18 and WideResNet34-10 · CIFAR-10, CIFAR-100, and TinyImageNet | AA accuracy: by 0.73%p to 1.66%p | Helped |
| use PeerAiD instead of adversarial distillation baselines | ResNet-18 and WideResNet34-10 · CIFAR-10, CIFAR-100, and TinyImageNet | clean accuracy: up to 4.72%p | Helped |
| add SWA to PeerAiD (ablation) | PeerAiD · TinyImageNet, ResNet-18 | AA accuracy: 20.00 -> 21.67 | Helped |
| use a WideResNet34-10 peer instead of a same-size teacher | PeerAiD with a ResNet-18 student · CIFAR-100 | AA accuracy: 26.99 AA | Helped |
| use PeerAiD rather than adversarial distillation baselines | MobileNetV2 · CIFAR-10 | AA accuracy: 50.02 vs 49.85 | Helped |
| train a peer tutor on student-generated adversarial examples instead of relying on a pretrained robust teacher (ablation) | peer network · CIFAR-100, ResNet-18 | robust accuracy against student-generated adversarial examples: 39.46 -> 69.19 | Helped |
Surprising findings
- The peer model is not robust at all against attacks on itself (0% PGD-20 accuracy) yet is highly robust against adversarial examples generated from the student.
- The pretrained robust teacher's robustness against student-generated adversarial examples quickly drops during distillation training.
Limitations
- Peer model is not robust to attacks aimed at itself, so it is specialized to the student and not a general robust model.
- Gains are modest on some settings (e.g., AutoAttack improvements of under 2 points) and vary across architectures.
Experimental setup
Datasets: CIFAR-10 CIFAR-100 TinyImageNet
Benchmarks: CIFAR-10 CIFAR-100 TinyImageNet
Metrics: clean accuracy FGSM accuracy PGD-20 accuracy AutoAttack accuracy
Baselines: PGD-AT TRADES AKD2 RSLAD IAD CAT AdaAD
Builds on: RSLAD IAD AKD2 TRADES online knowledge distillation
Closest papers by meaning
- Indirect Gradient Matching for Adversarial Robust Distillation (ICLR 2025 · similarity 0.87)
- Learning Better Certified Models from Empirically-Robust Teachers (NeurIPS 2026 · similarity 0.86)
- Improving Adversarial Robust Fairness via Anti-Bias Soft Label Distillation (NeurIPS 2024 · similarity 0.86)
- Out of Thin Air: Exploring Data-Free Adversarial Robustness Distillation (AAAI 2024 · similarity 0.86)
- AUTE: Peer-Alignment and Self-Unlearning Boost Adversarial Robustness for Training Ensemble Models (AAAI 2025 · similarity 0.86)
- A Closer Look at Curriculum Adversarial Training: From an Online Perspective (AAAI 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.