Research map / Optimization / Implicit bias and generalization
The Power of Interpolation: Understanding the Effectiveness of SGD in Modern Over-parametrized Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use mini-batch SGD with a fixed step size in the interpolation regime | Convex ERM with interpolated losses · Interpolated convex optimization | convergence rate | Helped |
| Increase batch size up to the critical batch size | Mini-batch SGD · Interpolated quadratic loss | iterations to target error | Helped |
| Increase batch size beyond the critical batch size | Mini-batch SGD · Interpolated quadratic loss | iterations to target error: 4 | No clear effect |
| Use batch size 1 instead of a larger batch under equal gradient-computation cost | Mini-batch SGD · Interpolated quadratic loss | computational cost to target accuracy | Helped |
| Increase the step size from the proposed near-optimal value to twice that value | Mini-batch SGD · Kernel learning experiments | convergence: 2 | Hurt |
| Use batch size m* instead of full gradient descent | SGD · MNIST, HINT-S, and TIMIT | training error convergence: 4 | Helped |
| Use very large batches well beyond m* | Mini-batch SGD · Training error versus gradient evaluations | computational efficiency | Hurt |
| Use an interpolating solution rather than a non-interpolating regime | SGD with convex losses · Interpolated convex optimization | convergence rate | Helped |
Closest papers by meaning
- Aiming towards the minimizers: fast convergence of SGD for overparametrized problems (NeurIPS 2023 · similarity 0.91)
- Fast convergence of stochastic subgradient method under interpolation (ICLR 2021 · similarity 0.90)
- Painless Stochastic Gradient: Interpolation, Line-Search, and Convergence Rates (NeurIPS 2019 · similarity 0.90)
- Fast Last-Iterate Convergence of SGD in the Smooth Interpolation Regime (NeurIPS 2025 · similarity 0.90)
- Tighter Convergence Bounds for Shuffled SGD via Primal-Dual Perspective (NeurIPS 2024 · similarity 0.88)
- Derivatives of Stochastic Gradient Descent in parametric optimization (NeurIPS 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.