Research map / Optimization / Implicit bias and generalization
Why Does Sharpness-Aware Minimization Generalize Better Than SGD?
Zixiang Chen, Junkai Zhang, Yiwen Kou, Xiangning Chen, Cho-Jui Hsieh, Quanquan Gu
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use SAM instead of SGD | WRN-16-8 · CIFAR-10 | top-1 accuracy (%): 96.69 -> 97.19 | Improved |
| use SAM instead of SGD | WRN-16-8 · CIFAR-100 | top-1 accuracy (%): 81.93 -> 83.68 | Improved |
| use SAM instead of SGD with Gaussian noise variance 0.1 | WRN-16-8 · CIFAR-10 noisy | top-1 accuracy (%): 95.87 -> 96.57 | Improved |
| use SAM instead of SGD with Gaussian noise variance 0.3 | WRN-16-8 · CIFAR-10 noisy | top-1 accuracy (%): 92.40 -> 93.37 | Improved |
| use SAM instead of SGD with Gaussian noise variance 1 | WRN-16-8 · CIFAR-10 noisy | top-1 accuracy (%): 79.50 -> 80.37 | Improved |
| start SAM at 10% of training with tau=0.02 (ablation) | ResNet-50 · ImageNet | top-1 accuracy (%): 76.4 -> 77.1 | Improved |
| start SAM at 90% of training with tau=0.02 (ablation) | ResNet-50 · ImageNet | top-1 accuracy (%): 76.4 -> 76.6 | Improved |
| start SAM at 10% instead of 90% of training with tau=0.02 (ablation) | ResNet-50 · ImageNet | top-1 accuracy (%): 76.6 -> 77.1 | Improved |
| use larger SAM tau=0.05 instead of 0.02 (ablation) | ResNet-50 · ImageNet | top-1 accuracy (%): 77.1 -> 76.2 | Hurt |
| use SAM with tau=0.05 started at 90% (ablation) | ResNet-50 · ImageNet | top-1 accuracy (%): 76.4 -> 76.2 | No clear effect |
Closest papers by meaning
- Friendly Sharpness-Aware Minimization (CVPR 2024 · similarity 0.93)
- Sharpness-Aware Minimization: General Analysis and Improved Rates (ICLR 2025 · similarity 0.93)
- Sharpness-Aware Minimization Efficiently Selects Flatter Minima Late In Training (ICLR 2025 · similarity 0.92)
- Sharpness-Aware Minimization Enhances Feature Quality via Balanced Learning (ICLR 2024 · similarity 0.92)
- Changing the Training Data Distribution to Reduce Simplicity Bias Improves In-distribution Generalization (NeurIPS 2024 · similarity 0.91)
- A Single-Step, Sharpness-Aware Minimization is All You Need to Achieve Efficient and Accurate Sparse Training (NeurIPS 2024 · similarity 0.91)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
