Research map / Optimization / Implicit bias and generalization
Unique Properties of Flat Minima in Deep Networks
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use identity initialization instead of random initialization (ablation) | SGD-trained fully connected ReLU networks · MNIST denoising | Hessian top eigenvalue | Helped |
| Increase the SGD step size (ablation) | Fully connected ReLU networks · MNIST denoising | Hessian top eigenvalue | Helped |
| Increase network depth (ablation) | SGD-trained fully connected ReLU networks · MNIST denoising | Hessian top eigenvalue | Hurt |
| Use SGD with a large step size instead of Adam with a small step size (ablation) | Six-layer fully connected ReLU network · MNIST denoising | Hessian top eigenvalue: roughly 6× | Helped |
| Use SGD with a large step size instead of Adam with a small step size (ablation) | Six-layer fully connected ReLU network · MNIST denoising | Intermediate gain | Helped |
| Compare minima using the line connecting them (ablation) | Flat and sharp minima of nonlinear networks · MNIST denoising | Apparent sharpness along interpolation line | Mixed |
Closest papers by meaning
- Exact Solutions of a Deep Linear Network (NeurIPS 2022 · similarity 0.91)
- Stable Minima of ReLU Neural Networks Suffer from the Curse of Dimensionality: The Neural Shattering Phenomenon (NeurIPS 2025 · similarity 0.91)
- Sharp Minima Can Generalize For Deep Nets (ICML 2017 · similarity 0.90)
- Deep linear networks for regression are implicitly regularized towards flat minima (NeurIPS 2024 · similarity 0.89)
- Flat Channels to Infinity in Neural Loss Landscapes (NeurIPS 2025 · similarity 0.89)
- Implicit bias of SGD in $L_2$-regularized linear DNNs: One-way jumps from high to low rank (ICLR 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.