Research map / Optimization / Optimizers
Trajectory of Mini-Batch Momentum: Batch Size Saturation and Convergence in High Dimensions
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use SGD+M with batch fraction at least the ICR | SGD+M on least squares · Least squares, large batch (ζ ≥ ICR) | convergence rate: 1-c/√κ | Helped |
| Increase the batch fraction while it remains below the ICR | SGD+M on least squares · Least squares, small batch (ζ ≤ ICR) | convergence rate: 1-cζ/κ̄ | Helped |
| Increase the batch fraction beyond the ICR | SGD+M on least squares · Least squares, large batch (ζ ≥ ICR) | convergence rate | No clear effect |
| Add momentum in the small-batch regime | Mini-batch SGD on least squares · Least squares, small batch (ζ ≤ ICR) | convergence rate: 1-cζ/κ̄ | No clear effect |
| Use the Volterra equation as the deterministic trajectory equivalent | SGD+M on high-dimensional least squares · Least squares, high-dimensional limit | loss trajectory approximation: n^{-C} | Helped |
Closest papers by meaning
- Momentum Further Constrains Sharpness at the Edge of Stochastic Stability (ICML 2026 · similarity 0.91)
- Perfect Parallelization in Mini-Batch SGD with Classical Momentum Acceleration (NeurIPS 2026 · similarity 0.91)
- High-dimensional limit theorems for SGD: Momentum and Adaptive Step-sizes (ICLR 2026 · similarity 0.90)
- A view of mini-batch SGD via generating functions: conditions of convergence, phase transitions, benefit from negative momenta. (ICLR 2023 · similarity 0.90)
- An Improved Analysis of Stochastic Gradient Descent with Momentum (NeurIPS 2020 · similarity 0.89)
- The Marginal Value of Momentum for Small Learning Rate SGD (ICLR 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.