Research map / Optimization / Implicit bias and generalization
Revisiting Sharpness-Aware Minimization: A More Faithful and Effective Implementation
Jianlong Chen, Zhiming Zhou
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace SAM gradient with explicit direction estimate (XSAM) | SAM · CIFAR-100 ResNet-18 | accuracy: 80.93 -> 81.24 | Improved |
| replace SAM gradient with explicit direction estimate (XSAM) | SAM · Tiny-ImageNet VGG-11 | accuracy: 51.96 -> 52.58 | Improved |
| replace SAM gradient with explicit direction estimate (XSAM) | SAM · ImageNet ResNet-50 | accuracy: 77.04 -> 77.22 | Improved |
| replace SAM gradient with explicit direction estimate (XSAM) | SAM · IWSLT2014 Transformer | BLEU: 35.30 -> 35.63 | Improved |
| replace SAM gradient with explicit direction estimate (XSAM) | SAM · CIFAR-100 ViT-Ti | accuracy: 67.80 -> 68.32 | Improved |
| add single-step ascent gradient (SAM) over SGD | SGD · CIFAR-100 ResNet-18 | accuracy: 78.55 -> 80.93 | Improved |
| increase ascent steps from k=1 to k=4 (ablation) | SAM · CIFAR-100 ResNet-18 | accuracy: 80.93 -> 80.65 | Hurt |
| increase ascent steps from k=1 to k=4 (ablation) | XSAM · CIFAR-100 ResNet-18 | accuracy: 81.27 -> 81.37 | Improved |
| increase ascent steps from k=1 to k=2 (ablation) | MSAM · CIFAR-100 ResNet-18 | accuracy: 80.93 -> 81.18 | Improved |
| add initial gradient g0 to multi-step sum (MSAM+ vs MSAM) (ablation) | MSAM · CIFAR-100 ResNet-18 k=2 | accuracy: 81.18 -> 80.86 | Hurt |
| add initial gradient g0 (LSAM+ vs LSAM) (ablation) | LSAM · CIFAR-100 ResNet-18 k=4 | accuracy: 80.74 -> 80.41 | Hurt |
| update alpha* at different frequencies (ablation) | XSAM · CIFAR-100 | accuracy | No clear effect |
Closest papers by meaning
- A Universal Class of Sharpness-Aware Minimization Algorithms (ICML 2024 · similarity 0.92)
- Sharpness-Aware Minimization Efficiently Selects Flatter Minima Late In Training (ICLR 2025 · similarity 0.91)
- Unveiling m-Sharpness Through the Structure of Stochastic Gradient Noise (NeurIPS 2025 · similarity 0.91)
- Sharpness-Aware Minimization: General Analysis and Improved Rates (ICLR 2025 · similarity 0.91)
- Lookbehind-SAM: k steps back, 1 step forward (ICML 2024 · similarity 0.91)
- Sharpness-Aware Minimization Enhances Feature Quality via Balanced Learning (ICLR 2024 · similarity 0.91)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
