Research map / Adversarial robustness and security / Certified and empirical robustness
Improving Robustness with Adaptive Weight Decay
Amin Ghiasi, Ali Shafahi, Reza Ardekani
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace weight decay with adaptive weight decay | PGD-7 adversarial training · CIFAR-10 | AutoAttack accuracy: 45.19 -> 50.03 | Improved |
| replace weight decay with adaptive weight decay | PGD-7 adversarial training · CIFAR-100 | AutoAttack accuracy: 22.53 -> 27.15 | Improved |
| replace weight decay with adaptive weight decay | PGD-7 adversarial training · Tiny ImageNet | AutoAttack accuracy: 16.73 -> 19.74 | Improved |
| replace weight decay with adaptive weight decay | PGD-7 adversarial training · Flowers | natural accuracy: 90.98 -> 90.39 | Hurt |
| replace weight decay with adaptive weight decay | PGD-7 adversarial training · CIFAR-100 | weight norm: 51.32 -> 13.41 | Improved |
| replace weight decay with adaptive weight decay | PGD-7 adversarial training · CIFAR-100 | non-adaptive total loss: 0.67 -> 1.51 | Hurt |
| add ASAM optimizer to adaptive weight decay (ablation) | AWD · CIFAR-100 WRN28-10 | AutoAttack accuracy: 27.15 -> 29.54 | Improved |
| add ASAM optimizer to standard weight decay (ablation) | Weight decay · CIFAR-100 WRN28-10 | AutoAttack accuracy: 22.53 -> 22.55 | No clear effect |
| use AWD + ASAM instead of TRADES etc | WRN32-10 · CIFAR-10 | AutoAttack accuracy: 53.40 -> 54.04 | Improved |
| train AWD for 50 instead of 100 epochs (ablation) | AWD + ASAM WRN32-10 · CIFAR-100 | AutoAttack accuracy: 29.70 -> 29.25 | No clear effect |
| train AWD for 5 epochs instead of 100 (ablation) | AWD + ASAM WRN32-10 · CIFAR-100 | AutoAttack accuracy: 29.70 -> 11.03 | Hurt |
| use AWD with ResNet50 natural training | ResNet50 · CIFAR-100 | training accuracy: 98.57 -> 95.29 | Hurt |
Closest papers by meaning
- Relating Adversarially Robust Generalization to Flat Minima (ICCV 2021 · similarity 0.90)
- Probably Approximately Global Robustness Certification (ICML 2025 · similarity 0.88)
- Why Do We Need Weight Decay in Modern Deep Learning? (NeurIPS 2024 · similarity 0.87)
- Improving Accuracy-robustness Trade-off via Pixel Reweighted Adversarial Training (ICML 2024 · similarity 0.87)
- Doubly Robust Instance-Reweighted Adversarial Training (ICLR 2024 · similarity 0.87)
- Rethinking Weight Decay for Robust Fine-Tuning of Foundation Models (NeurIPS 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
