Research map / Optimization / Implicit bias and generalization
The Implicit Regularization of Stochastic Gradient Flow for Least Squares
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Stop stochastic gradient flow by balancing bias and variance instead of optimizing ridge's tuning parameter | Stochastic gradient flow versus ridge · Synthetic least squares (n=100, p=500, m=20) | risk ratio to optimally stopped ridge: 1.0032 | Mixed |
| Use state-dependent gradient-noise covariance instead of constant covariance (ablation) | Stochastic gradient flow · Univariate responseless least squares | limiting variance: epsilon/(2m) versus variance tending to zero | Helped |
| Increase mini-batch size or decrease step size | Stochastic gradient flow excess-risk bound relative to ridge · Least squares, all t > 0 | excess-risk bound | Helped |
| Use a constant step size with nonzero mini-batching noise rather than noiseless gradient flow | Stochastic gradient flow · Underparametrized least squares | limiting optimization error | Hurt |
Closest papers by meaning
- The Benefits of Implicit Regularization from SGD in Least Squares Problems (NeurIPS 2021 · similarity 0.90)
- Implicit Regularization or Implicit Conditioning? Exact Risk Trajectories of SGD in High Dimensions (NeurIPS 2022 · similarity 0.89)
- Controlling the Flow: Stability and Convergence for Stochastic Gradient Descent with Decaying Regularization (NeurIPS 2025 · similarity 0.89)
- Implicit Bias of SGD for Diagonal Linear Networks: a Provable Benefit of Stochasticity (NeurIPS 2021 · similarity 0.89)
- Sampling from Gaussian Process Posteriors using Stochastic Gradient Descent (NeurIPS 2023 · similarity 0.88)
- Nonasymptotic Analysis of Stochastic Gradient Descent with the Richardson–Romberg Extrapolation (ICLR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.