Research map / Optimization / Nonconvex and smooth optimization
Acceleration and Nesterov dynamics
Studies accelerated gradient methods through geometric, continuous-time, and physical-system views.
91 accepted papers at ICML, NeurIPS, ICLR, CVPR and AAAI (2016–2026) · an approach in Nonconvex and smooth optimization, keyword cluster “coordinate · proximal · bounds”
State History Tried Checklist Preprints Who Export
Where it stands
Not enough benchmark data Fewer than four papers report the same benchmark, so there is no record to follow.
19 papers in 2023–24 and 20 in 2025–26 (at ICML, NeurIPS, ICLR, CVPR and AAAI); its share of the topic held steady, from 5.9% to 5.7%.
How this is decided
A benchmark is tracked when at least four papers of this approach report their own method on the same dataset and metric in at least two different years. Its record is the best number reported up to 2024 compared with the best reported in 2025–26: still improving means at least 1% better, flat means at most 0.2% better with at least two papers from 2025–26 trying. Numbers on a different scale from the rest (a fraction among percentages) and far outliers are left out. Results come from different papers with different setups, so they are signals, not head-to-head comparisons. The share of papers is counted at ICML, NeurIPS, ICLR, CVPR and AAAI.
How it got here
Most cited paper of each year
- 2016 · Even Faster Accelerated Coordinate Descent Using Non-Uniform Sampling (ICML 2016 · 100 citations)
- 2017 · Gradient Descent Can Take Exponential Time to Escape Saddle Points (NeurIPS 2017 · 63 citations)
- 2018 · Direct Runge-Kutta Discretization Achieves Acceleration (NeurIPS 2018 · 55 citations)
- 2019 · Riemannian Adaptive Optimization Methods (ICLR 2019 · 145 citations)
- 2020 · Conformal Symplectic and Relativistic Optimization (NeurIPS 2020 · 19 citations)
- 2021 · On Riemannian Optimization over Positive Definite Matrices with the Bures-Wasserstein Geometry (NeurIPS 2021 · 9 citations)
- 2022 · Breaking the Convergence Barrier: Optimization via Fixed-Time Convergent Flows (AAAI 2022 · 13 citations)
- 2023 · Continuous-time Analysis of Anchor Acceleration (NeurIPS 2023 · 3 citations)
- 2024 · Decentralized Riemannian Conjugate Gradient Method on the Stiefel Manifold (ICLR 2024 · 1 citation)
- 2025 · Accelerated Over-Relaxation Heavy-Ball Method: Achieving Global Accelerated Convergence with Broad Generalization (ICLR 2025 · 1 citation)
- 2026 · Accelerated Multiple Wasserstein Gradient Flows for Multi-objective Distributional Optimization (ICML 2026 · 0 citations)
Earliest papers here
- Even Faster Accelerated Coordinate Descent Using Non-Uniform Sampling (ICML 2016 · 100 citations)
- Adaptive Averaging in Accelerated Descent Dynamics (NeurIPS 2016 · 14 citations)
- Gradient Descent Can Take Exponential Time to Escape Saddle Points (NeurIPS 2017 · 63 citations)
What has been tried
70 papers here report 437 measured changes (a component added, removed or swapped, compared with a baseline): helped 69%, no clear effect 14%, hurt 13%, mixed 5%. Papers mostly publish changes that helped, so "helped" is over-represented.
Ablations in recent papers
| What was changed | Measured on | Result | Effect | Paper |
|---|---|---|---|---|
| treat the coupling implicitly instead of using the explicit coupling scheme | strongly-convex-strongly-concave saddle-point problems · convergence rate | 1-\mathcal O(\sqrt{\rho}) | Helped | Accelerated Over-Relaxation Heavy-Ball Method: Achieving Global Accelerated Convergence with Broad Generalization (ICLR 2025 · 1 citation) |
| Change α while holding η values fixed | MNIST, batch size 60, six epochs · training loss | Mixed | Nesterov acceleration despite very noisy gradients (NeurIPS 2024 · 1 citation) | |
| Replace the optimal inner mapping x_2^*(x_1)=x_1 with the constant mapping that fixes x_2 | Illustrative quadratic example · smoothness constant | null | Hurt | Sharper Convergence Rates for Nonconvex Optimisation via Reduction Mappings (NeurIPS 2025 · 0 citations) |
| Set α̂=0.01 instead of using the exact α | Quadratic minimization, κ=10^7 · convergence | Hurt | Hamiltonian Descent Algorithms for Optimization: Accelerated Rates via Randomized Integration Time (NeurIPS 2025 · 0 citations) | |
| use RCDlin instead of RCD | PCA · convergence rate | Helped | Riemannian coordinate descent algorithms on matrix manifolds (ICML 2024 · 0 citations) | |
| Omit the burn-in heuristic | WordNet mammals' subtree embedding in the Poincaré ball · mean average precision | Hurt | Learning-Rate-Free Stochastic Optimization over Riemannian Manifolds (ICML 2024 · 0 citations) | |
| Set γ1=γ2=1 (block exact minimization) instead of optimal BGD stepsizes | Least-squares · spectral radius | ρ(M(1, 1)) = λ1(C Cᵀ) | Hurt | Block Acceleration Without Momentum: On Optimal Stepsizes of Block Gradient Descent for Least-Squares (ICML 2024 · 0 citations) |
| use RGD with step size 1/L instead of RGD with step size 1/(Lζ₍O(R)₎) | Karcher mean on hyperbolic space · function value | Helped | Convergence and Trade-Offs in Riemannian Gradient Descent and Riemannian Proximal Point (ICML 2024 · 0 citations) | |
| Add the gradient-difference term to Lie Heavy-Ball's momentum update | Locally geodesically strongly convex Lie-group objectives · convergence rate | \left(1+\frac{\mu}{16L}\right)^{-1} -> \left(1+\frac{1}{30}\sqrt{\mu}\min\left\{\frac{1}{\sqrt{2L}}, \frac{1}{2p(a)}\rig | Helped | Quantitative Convergences of Lie Group Momentum Optimizers (NeurIPS 2024 · 0 citations) |
| Use decreasing rather than constant step sizes | Elastic net and data poisoning · Jacobian-vector approximation error | Helped | Nonsmooth Implicit Differentiation: Deterministic and Stochastic Convergence Rates (ICML 2024) |
What reviewers will expect
Datasets and baselines named in the 11 papers of this approach from 2025–2026 that have a full-text summary. Leaving out the common ones is a frequent review question.
Baselines
- GD · 2 of 11 papers
- ADAM · 2 of 11 papers
- gradient descent · 2 of 11 papers
Recent preprints
Loading recent arXiv preprints…
Who publishes on it
Institutions on the most papers of this approach since 2023 (of 39 papers; from OpenAlex and the papers' affiliations).
- Seoul National University · 3 papers
- Texas A&M University - College Station · 3 papers
- Yale University · 2 papers
- Stockholm University · 2 papers
- Google · 2 papers
Use it
Download BibTeX of 40 key papersOpen the topic in the interactive map
Get alerts. Describe your idea in scime atlas and save it (free scime account): you get a weekly email when new preprints land close to it.
Related approaches
Same keyword cluster
- Adaptive first-order convex methods (190 papers)
- Stochastic gradient last-iterate analysis (178 papers)
- Coordinate and Adam convergence analysis (188 papers)
- Langevin sampling for nonconvex problems (70 papers)
Elsewhere in the topic
- Nonconvex minimax and constrained methods (100 papers)
- Stochastic convex optimization complexity (94 papers)
- Equilibrium learning in zero-sum games (88 papers)
- Stochastic variance reduction (86 papers)
- Complexity of nonconvex first-order methods (79 papers)
- Penalty and implicit-gradient bilevel methods (65 papers)
- Proximal finite-sum methods (47 papers)
- Stochastic Frank-Wolfe methods (43 papers)
Benchmark numbers, ablations and baseline numbers are extracted by AI from each paper's full text and can be wrong; the paper is the authority. Data: results index of 2026-11-02, citation counts from OpenAlex (2026-10-09).