Research map / Adversarial robustness and security / Certified and empirical robustness
On the Duality Between Sharpness-Aware Minimization and Adversarial Training
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Shows that SAM alone, without adversarial examples, improves adversarial robustness and clean accuracy over standard training, with theoretical analysis via a robust/non-robust feature toy model linking SAM to AT.
Problem
Adversarial training improves robustness but reduces clean accuracy; SAM was designed for clean generalization and its effect on adversarial robustness was unexplored.
Approach
Views SAM (weight perturbation) and AT (input perturbation) as duals; argues via a robust/non-robust feature binary model that SAM biases the classifier toward robust features (W_R^SAM > W_R^*) while keeping natural data, so it improves robustness without the clean-accuracy loss of AT.
Key results
- On CIFAR-100 with l2 eps=32/255 AutoAttack, SAM (rho=0.4) reaches 25.4% robust accuracy and 78.7% clean accuracy versus 3.4% and 76.0% for Adam.
- On CIFAR-10 SAM (rho=0.4) gives 51.9% average robustness versus 21.3% for Adam and 37.8% for SGD, while clean accuracy stays at 94.7-95.5% vs 94.5% for SGD.
- l_inf-AT (eps=8/255) reduces CIFAR-10 clean accuracy to 84.5%, while SAM keeps clean accuracy above standard training; AT still gives higher robustness (AA 79.5%).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use SAM instead of SGD | PreActResNet-18 · CIFAR-10 | average robustness: 37.8 -> 51.9 | Helped |
| Use SAM instead of SGD | PreActResNet-18 · CIFAR-100 | average robustness: 15.1 -> 27.6 | Helped |
| Use SAM instead of SGD | PreActResNet-18 · CIFAR-100 | natural accuracy: 76.5 -> 78.7 | Helped |
| Use SAM instead of SGD | PreActResNet-18 · Tiny ImageNet | AA. accuracy: 2.5 -> 10.4 | Helped |
| Use SAM instead of SGD | PreActResNet-18 · CIFAR-10C | corruption average accuracy: 34.91 -> 32.60 | Hurt |
| Use adaptive SAM (ASAM) instead of standard SAM | PreActResNet-18 · CIFAR-10C | corruption average accuracy: 34.91 -> 36.69 | Helped |
| Use SAM instead of standard training | DeepLabv3, ResNet-50 · Stanford background | natural mIoU: 64.0 -> 64.8 | Helped |
| Use SAM instead of standard training | DeepLabv3, MobileNetV2 · VOC2012 validation | natural mIoU: 66.9 -> 68.6 | Helped |
| Use SAM instead of ST (AdamW) | distilbert-base-uncased · Rotten Tomatoes, PSO | attack success rate: 22.7 -> 17.1 | Helped |
| Use SAM instead of ST (AdamW) | distilbert-base-uncased · Rotten Tomatoes, IGA | attack success rate: 84.1 -> 74.5 | Helped |
| Increase SAM ρ from 0.1 to 0.4 (ablation) | PreActResNet-18 · CIFAR-10 | average robustness: 43.4 -> 51.9 | Helped |
| Increase SAM ρ from 0.1 to 0.4 (ablation) | PreActResNet-18 · CIFAR-100 | average robustness: 21.3 -> 27.6 | Helped |
Surprising findings
- SAM alone improves adversarial robustness substantially over standard training, without any adversarial examples and without losing clean accuracy.
- Even tiny AT budgets (eps=1/255) reduce clean accuracy on CIFAR-100 by about 3.8%.
Limitations
- Theory is analyzed only on a simple binary linear model and does not directly generalize to multi-layer networks.
- A sizable robustness gap remains between SAM and adversarial training under strong attacks.
Experimental setup
Datasets: CIFAR-10 CIFAR-100 Tiny-ImageNet Stanford Background VOC2012 Rotten Tomatoes
Benchmarks: CIFAR-10 CIFAR-100 Tiny-ImageNet CIFAR-10C Stanford Background VOC2012 Rotten Tomatoes
Metrics: natural accuracy robust accuracy AutoAttack accuracy mIoU attack success rate
Baselines: SGD Adam l_inf-AT l_2-AT AWP ASAM ESAM FastAT
Builds on: Sharpness-Aware Minimization Adversarial Training Adversarial Weight Perturbation robust and non-robust features
Closest papers by meaning
- Sharpness-Aware Minimization Efficiently Selects Flatter Minima Late In Training (ICLR 2025 · similarity 0.90)
- Friendly Sharpness-Aware Minimization (CVPR 2024 · similarity 0.90)
- Adversarial Training Can Provably Improve Robustness: Theoretical Analysis of Feature Learning Process Under Structured Data (ICLR 2025 · similarity 0.89)
- Robust SAM: On the Adversarial Robustness of Vision Foundation Models (AAAI 2025 · similarity 0.88)
- Why is SAM Robust to Label Noise? (ICLR 2024 · similarity 0.88)
- Exploring Adversarial Robustness of Deep State Space Models (NeurIPS 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.