Research map / Optimization / Implicit bias and generalization
When Do Flat Minima Optimizers Work?
Jean Kaddour, Linqing Liu, Ricardo Silva, Matt Kusner
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Average SAM iterates to form WASAM | SAM · 42 benchmark experiments | generalization performance: 39/42 cases | Improved |
| Use SAM instead of the non-flat baseline | NLP tasks · NLP experiments | task performance: 7/10 experiments | Improved |
| Use SWA instead of the non-flat baseline | NLP tasks · NLP experiments | task performance: 1/10; 4 tasks | Mixed |
| Use SWA instead of the non-flat baseline | Graph representation learning · GRL experiments | task performance: 10/12 experiments | Improved |
| Use SAM instead of the non-flat baseline | Graph representation learning · GRL experiments | task performance: 4/12 experiments | Mixed |
| Use flat-minima optimizers instead of the baseline | SAGE or DGCN · OGB-Proteins | ROC-AUC | No clear effect |
| Use flat-minima optimizers instead of the baseline | SAGE or DGCN · OGB-Products | accuracy | Improved |
| Replace GCN layers with SAGE layers | Citation2 link prediction · OGB-Citation2 | MRR: >1.15%; -0.31% | Mixed |
| Average SAM iterates to form WASAM | SAM · WRN on CIFAR100 and GIN on Code2 | median dominant Hessian eigenvalue: 673 -> 117; 16.65 -> 9.96 | Improved |
| Use SAM with AdamW instead of the AdamW baseline | ResNet34 · CIFAR100 | accuracy: -2.29 | Hurt |
| Switch the base optimizer from Adam to RMSprop (ablation) | SWA and SAM · GIN on Code2 | F1: +0.83_{7 0.11} -> +0.67; +0.57_{7 0.09} -> +0.70 | Mixed |
| Use data augmentation rather than none (ablation) | SWA and SAM · ResNet18 on CIFAR100 | accuracy improvement: None: +2.23, +2.20; Basic: +0.13, +0.74; AA: +3.03, +3.35 | Mixed |
Closest papers by meaning
- Flat Minima and Generalization: Insights from Stochastic Convex Optimization (ICML 2026 · similarity 0.91)
- Sharpness-Aware Minimization Efficiently Selects Flatter Minima Late In Training (ICLR 2025 · similarity 0.90)
- Sharpness-Aware Minimization Enhances Feature Quality via Balanced Learning (ICLR 2024 · similarity 0.90)
- Sharpness-Aware Minimization: General Analysis and Improved Rates (ICLR 2025 · similarity 0.89)
- Improving Sharpness-Aware Minimization by Lookahead (ICML 2024 · similarity 0.88)
- A Unified Stability Analysis of SAM vs SGD: Role of Data Coherence and Emergence of Simplicity Bias (NeurIPS 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
