Research map / Adversarial robustness and security / Certified and empirical robustness
Bag of Tricks for Adversarial Training
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Increase TRADES weight decay from 2×10⁻⁴ to 5×10⁻⁴ (ablation) | TRADES on WRN-34-10 · CIFAR-10, ℓ∞ ε=8/255 | AutoAttack accuracy: 50.94 -> 53.80 | Helped |
| Use eval-mode BN while crafting training adversarial examples (ablation) | PGD-AT · CIFAR-10 | Clean accuracy: 86.07 -> 87.38 | Helped |
| Early stop the adversarial attack intensity during training (ablation) | PGD-AT · CIFAR-10 | AutoAttack accuracy: 48.51 -> 46.6 | Hurt |
| Warm up the learning rate (ablation) | PGD-AT · CIFAR-10 | AutoAttack accuracy: 48.51 -> 48.26 | No clear effect |
| Warm up the perturbation magnitude (ablation) | PGD-AT · CIFAR-10 | AutoAttack accuracy: 48.51 -> 48.44 | No clear effect |
| Replace ReLU with Softplus (ablation) | PGD-AT on ResNet-18 · CIFAR-10 | PGD-10 accuracy: 53.58 -> 54.30 | Helped |
| Use Adam instead of SGD with momentum (ablation) | PGD-AT · CIFAR-10 | AutoAttack accuracy: 48.51 -> 44.04 | Hurt |
| Increase label smoothing moderately (ablation) | PGD-AT on ResNet-18 · CIFAR-10, PGD-1000 C&W objective | accuracy: 50.64 -> 51.07 | Helped |
| Use excessive label smoothing rather than moderate smoothing (ablation) | PGD-AT on ResNet-18 · CIFAR-10, PGD-1000 C&W objective | accuracy: 51.07 -> 50.54 | Hurt |
| Increase weight decay from 1×10⁻⁴ to 5×10⁻⁴ (ablation) | PGD-AT · CIFAR-10 | robust accuracy: up to ~7% | Helped |
| Apply linear learning-rate scaling when changing batch size (ablation) | PGD-AT · CIFAR-10 | robust accuracy | Helped |
Closest papers by meaning
- Robust Models are less Over-Confident (NeurIPS 2022 · similarity 0.92)
- Adversarial Training Can Provably Improve Robustness: Theoretical Analysis of Feature Learning Process Under Structured Data (ICLR 2025 · similarity 0.92)
- Adversarial Robustness: From Self-Supervised Pre-Training to Fine-Tuning (CVPR 2020 · similarity 0.92)
- Once-for-All Adversarial Training: In-Situ Tradeoff between Robustness and Accuracy for Free (NeurIPS 2020 · similarity 0.91)
- On the Scalability of Certified Adversarial Robustness with Generated Data (NeurIPS 2024 · similarity 0.91)
- A Closer Look at Curriculum Adversarial Training: From an Online Perspective (AAAI 2024 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.