Research map / Optimization / Nonconvex and smooth optimization
On the Convergence of Nesterov’s Accelerated Gradient Method in Stochastic Settings
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use Nesterov ASG with the standard accelerated parameters | Smooth strongly-convex quadratic objectives with unbiased, independent bounded-variance gradients · Stochastic approximation | convergence rate: ((√Q - 1)/√Q + ε_k)^(2k) versus ((Q - 1)/(Q + 1))^(2k) | Helped |
| Use standard Nesterov step-size and momentum instead of parameters ensuring R(α, β) < 1 | Finite-sum ASG on smooth strongly-convex quadratic component functions · Finite-sum | expected distance to optimum: lim_{k → ∞} E[‖y_k - x*‖] = ∞ | Hurt |
| Use parameters satisfying R(α, β) < 1 | ASG on smooth strongly-convex finite sums · Finite-sum | convergence: R(α, β)^k | Helped |
| Set momentum β to 0 (ablation) | ASG on smooth strongly-convex objectives · Stochastic approximation | convergence rate: ((Q - 1)/(Q + 1))^(2k) versus ((√Q - 1)/√Q + ε_k)^(2k) | Hurt |
| Set momentum β to 0 and use α = 2/(μ + L) (ablation) | ASG on smooth strongly-convex finite sums · Finite-sum | convergence rate: ((Q - 1)/(Q + 1))^k | Mixed |
Closest papers by meaning
- A Universally Optimal Multistage Accelerated Stochastic Gradient Method (NeurIPS 2019 · similarity 0.91)
- Nesterov acceleration despite very noisy gradients (NeurIPS 2024 · similarity 0.90)
- Gradient correlation is a key ingredient to accelerate SGD with momentum (ICLR 2025 · similarity 0.90)
- Nesterov acceleration in benignly non-convex landscapes (ICLR 2025 · similarity 0.89)
- Advancing the Lower Bounds: an Accelerated, Stochastic, Second-order Method with Optimal Adaptation to Inexactness (ICLR 2024 · similarity 0.88)
- Nonasymptotic Analysis of Stochastic Gradient Descent with the Richardson–Romberg Extrapolation (ICLR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.