Research map / Optimization / Implicit bias and generalization
From Gradient Flow on Population Loss to Learning with Stochastic Gradient Descent
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Derive an admissible potential from an admissible gradient-flow convergence rate | GD and SGD on objectives satisfying self-bounding regularity · General non-convex objectives | expected suboptimality: O \prn[\big]{\tfrac{1}{\sqrt{T}}}; O\prn[\big]{\tfrac{1}{T}} | Helped |
| Add noise regularity to the admissible-potential and self-bounding conditions | SGD · General non-convex objectives | expected suboptimality: \wt{O}\prn[\big]{\tfrac{1}{\sqrt{T}}} | Helped |
| Use the admissible potential induced by the KŁ gradient-flow rate | GD · Smooth KŁ objectives | expected suboptimality: \lesssim \prn*{\frac{H \Fkl(w_0)}{\alpha}}^{{1}/{1 + \theta}} \cdot \frac{1}{T^{1/(2 + 2 \theta)}} | Helped |
| Use the admissible potential induced by the KŁ gradient-flow rate with constant noise scale | SGD · Smooth KŁ objectives | expected suboptimality: \lesssim \prn*{\frac{BH^3 \Fkl(w_0)}{\alpha^2 T} }^{1/2 + 2 \theta} | Helped |
| Apply the potential framework to the phase-retrieval objective | GD · Phase retrieval | population loss: O\prn*{\min\crl*{\frac{1}{T}, e^{- O(T- t_0)}}} | Helped |
| Apply the potential framework to phase retrieval with noise regularity | SGD · Phase retrieval | population loss: \wt O\prn*{\tfrac{1}{\sqrt{T}}} | Helped |
| Apply the potential framework from an initialization satisfying h_ms(W_0) > 0 | GD · Matrix square root | population loss: O\prn*{\min\crl*{\frac{1}{T}, e^{- O(T- t_0)}}} | Helped |
| Apply the potential framework from an initialization satisfying h_ms(W_0) > 0 with noise regularity | SGD · Matrix square root | population loss: \wt{O}\prn*{ \frac{1}{\sqrt{T}}} | Helped |
| Remove the additional self-bounding regularity conditions for GD | GD with an admissible potential · Convex counterexample constructed for any T_0 | steps to find a 1/10-suboptimal solution: t \le T_0 | Hurt |
| Use fresh-sample phase-retrieval stochastic gradients with χ(z) = min{√z, c} | SGD · Phase retrieval | population loss: 1/\sqrt{T} | No clear effect |
| Replace trajectory-distance analysis with admissible-potential analysis | GD and SGD · Non-convex optimization | expected suboptimality | Helped |
Closest papers by meaning
- The Global Convergence Time of Stochastic Gradient Descent in Non-Convex Landscapes: Sharp Estimates via Large Deviations (ICML 2025 · similarity 0.93)
- Fast Last-Iterate Convergence of SGD in the Smooth Interpolation Regime (NeurIPS 2025 · similarity 0.91)
- A Precise Characterization of SGD Stability Using Loss Surface Geometry (ICLR 2024 · similarity 0.90)
- Controlling the Flow: Stability and Convergence for Stochastic Gradient Descent with Decaying Regularization (NeurIPS 2025 · similarity 0.90)
- On the Unstable Convergence Regime of Gradient Descent (AAAI 2024 · similarity 0.90)
- Statistical Guarantees for High-Dimensional Stochastic Gradient Descent (NeurIPS 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.