Research map / Optimization / Implicit bias and generalization
Stabilizing Sharpness-Aware Minimization Through A Simple Renormalization Strategy
Chengli Tan, Jiangshe Zhang, Junmin Liu, Yicheng Wang, Yunda Hao
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add gradient renormalization (SSAM) | SAM · ImageNet-1K val, ViT-S-32 | top-1 accuracy (%): 69.98 -> 71.15 | Improved |
| add gradient renormalization (SSAM) | SAM · ImageNet-1K val, ViT-S-16 | top-1 accuracy (%): 76.88 -> 77.41 | Improved |
| add gradient renormalization (SSAM) | SAM · ImageNet-1K val, ResNet-18 | top-1 accuracy (%): 70.66 -> 70.76 | Improved |
| add gradient renormalization (SSAM) | SAM · ImageNet-1K val, ResNet-50 | top-1 accuracy (%): 77.82 -> 77.89 | Improved |
| add gradient renormalization (SSAM) | SAM · CIFAR-10, PyramidNet-110 | test accuracy (%): 96.91 -> 97.04 | Improved |
| add gradient renormalization (SSAM) | SAM · CIFAR-100 (5 backbones) | test accuracy (%): ResNet-20 70.77 -> 70.48; WRN-28-10 82.94 -> 82.80 | Mixed |
| add gradient renormalization (SSAM) | SAM · Quartic saddle toy function, large learning rate | rate of successful training | Improved |
| add gradient renormalization (SSAM) | SAM · Quadratic loss d=20 | convergence speed | Hurt |
| add gradient renormalization (SSAM) | SAM · ResNet-20 CIFAR-10 / ResNet-56 CIFAR-100 | top Hessian eigenvalue (sharpness) | Improved |
| add gradient norm normalization in SAM (SAM* vs SAM) | SAM · CIFAR-10 and CIFAR-100 | test accuracy (%): CIFAR-10 WRN-28-10 96.55 -> 96.91 | No clear effect |
| add sharpness-aware ascent step (SAM instead of SGD) | SGD · CIFAR-10, ResNet-20 | test accuracy (%): 92.78 -> 93.43 | Improved |
| use GASAM instead of SAM | SAM · CIFAR-10, PyramidNet-110 | test accuracy (%): 96.91 -> 81.83 | Hurt |
Closest papers by meaning
- Sharpness-Aware Minimization Efficiently Selects Flatter Minima Late In Training (ICLR 2025 · similarity 0.92)
- Unveiling m-Sharpness Through the Structure of Stochastic Gradient Noise (NeurIPS 2025 · similarity 0.92)
- Sharpness-Aware Minimization: General Analysis and Improved Rates (ICLR 2025 · similarity 0.91)
- Tilted Sharpness-Aware Minimization (ICML 2025 · similarity 0.91)
- Explicit Eigenvalue Regularization Improves Sharpness-Aware Minimization (NeurIPS 2024 · similarity 0.91)
- $\boldsymbol{\mu}\mathbf{P^2}$: Effective Sharpness Aware Minimization Requires Layerwise Perturbation Scaling (NeurIPS 2024 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
