Research map / Optimization / Nonconvex and smooth optimization
Nonlinear Acceleration of Stochastic Algorithms
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Apply regularized nonlinear acceleration (RNA) to k=10 SGD iterates with restarts | SGD on least squares · Synthetic least squares, d=500 | f(x)-f(x*) at early iterations | Helped |
| Apply RNA to SGD iterates | SGD on least squares · Synthetic least squares, high noise | asymptotic convergence | No clear effect |
| Apply RNA to SGD iterates | SGD on finite-sum problem (Sido0) · Sido0 bad conditioning | convergence to optimum | No clear effect |
| Apply RNA to Katyusha iterates | Katyusha · Finite sums, Sido0 bad conditioning | convergence speed | No clear effect |
| Apply RNA to SAGA and SVRG iterates | SAGA, SVRG · Finite sums, Sido0 bad conditioning | convergence speed | Mixed |
| Wait several data passes (about N queries) before adding a snapshot to the RNA sequence | RNA on finite-sum methods · Finite sums | extrapolation cost | No clear effect |
Closest papers by meaning
- Nonasymptotic Analysis of Stochastic Gradient Descent with the Richardson–Romberg Extrapolation (ICLR 2025 · similarity 0.89)
- Revisiting the Last-Iterate Convergence of Stochastic Gradient Methods (ICLR 2024 · similarity 0.88)
- Stochastic Newton Proximal Extragradient Method (NeurIPS 2024 · similarity 0.88)
- Derivatives of Stochastic Gradient Descent in parametric optimization (NeurIPS 2024 · similarity 0.87)
- Faster Stochastic Optimization with Arbitrary Delays via Adaptive Asynchronous Mini-Batching (ICML 2025 · similarity 0.87)
- Direct Acceleration of Stochastic Root-Finding Without Variance Reduction and Regularization (NeurIPS 2026 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.