Research map / Optimization / Implicit bias and generalization
Gradient Descent Monotonically Decreases the Sharpness of Gradient Flow Solutions in Scalar Networks and Beyond
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Gradient descent on scalar linear networks (depth D, squared loss) with initialization in the positive invariant set S_G | GD iterates · scalar linear networks | GFS sharpness over iterations | Helped |
| Gradient descent with GFS sharpness crossing below 2/eta while loss is sufficiently small (loss <= delta^2/200) (ablation) | GD on scalar networks · scalar linear networks | final sharpness | No clear effect |
| Gradient descent on squared regression model, GFS sharpness monotonic decrease check across 100 experiments | GD on squared regression · squared regression model | normalized GFS sharpness change: max normalized change 0.00104; ~78% of steps negative | Helped |
| Gradient descent on realistic networks (FC, VGG11-BN, ResNet20) with MSE loss | GD on CIFAR-10 subsets · CIFAR-10 subset, FC/VGG11/ResNet20 | GFS sharpness trajectory | Helped |
| Occasional slight increase in GFS sharpness early in optimization on realistic networks | GD on realistic networks · CIFAR-10 subset | GFS sharpness monotonicity | Mixed |
Closest papers by meaning
- Analyzing Sharpness along GD Trajectory: Progressive Sharpening and Edge of Stability (NeurIPS 2022 · similarity 0.92)
- On the Unstable Convergence Regime of Gradient Descent (AAAI 2024 · similarity 0.91)
- Convergence Rates for Gradient Descent on the Edge of Stability for Overparametrised Least Squares (NeurIPS 2025 · similarity 0.90)
- A Minimalist Example of Edge-of-Stability and Progressive Sharpening (NeurIPS 2025 · similarity 0.89)
- A Precise Characterization of SGD Stability Using Loss Surface Geometry (ICLR 2024 · similarity 0.89)
- Large Stepsize Gradient Descent for Non-Homogeneous Two-Layer Networks: Margin Improvement and Fast Optimization (NeurIPS 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.