Nonconvex and smooth optimization: research map
1,707 papers on Nonconvex and smooth optimization in Optimization, from ICML, NeurIPS, ICLR, CVPR, AAAI, ICCV and ECCV, and the journals TPAMI, IJCV and JMLR (2016–2026), grouped into 5 clusters and 18 approaches. The busiest year so far is 2025.
What it is
The mathematics of optimisation methods: convergence rates and complexity bounds for gradient methods on nonconvex, minimax and bilevel problems, variance reduction, acceleration and projection-free methods such as Frank-Wolfe.
Terms you will see on this page:
- Stationary point
- a point where the gradient is near zero, the usual goal in nonconvex problems.
- Minimax problem
- optimising one variable against another, as in GANs or games.
- Variance reduction
- techniques that make stochastic gradients less noisy.
Start reading with the most cited papers here:
- SGDR: Stochastic Gradient Descent with Warm Restarts (ICLR 2017 · 1,699 citations)
- SPIDER: Near-Optimal Non-Convex Optimization via Stochastic Path-Integrated Differential Estimator (NeurIPS 2018 · 243 citations)
Within Optimization, its share held steady from 36.9% in 2023–24 to 36.0% in 2025–26 (320 → 352 papers at ICML, NeurIPS, ICLR, CVPR and AAAI, the venues with data for all four years).
Explore Nonconvex and smooth optimization in the interactive map
Working on something in this topic? Describe your idea in scime atlas to see which approach it falls under, the closest papers by meaning and how crowded the spot has become.
What changed recently
Approaches whose share of the topic moved most between 2023–24 and 2025–26 (ICML, NeurIPS, ICLR, CVPR and AAAI):
- Stochastic gradient last-iterate analysis rising: 10% → 13% of the topic (33 → 47 papers)
- Langevin sampling for nonconvex problems rising: 4% → 7% of the topic (14 → 25 papers)
- Coordinate and Adam convergence analysis rising: 12% → 14% of the topic (39 → 49 papers)
- Complexity of nonconvex first-order methods falling: 8% → 5% of the topic (25 → 18 papers)
- Extragradient for variational inequalities falling: 4% → 2% of the topic (14 → 8 papers)
Papers from 2025–2026 that are already cited most:
- Improved Rates of Differentially Private Nonconvex-Strongly-Concave Minimax Optimization (AAAI 2025 · 3 citations)
- Rewind-to-Delete: Certified Machine Unlearning for Nonconvex Functions (NeurIPS 2025 · 2 citations)
- Beyond Monotonicity: On the Convergence of Learning Algorithms in Standard Auction Games (AAAI 2025 · 2 citations)
- Langevin Monte Carlo Beyond Lipschitz Gradient Continuity (AAAI 2025 · 1 citations)
Approaches and key papers
First-order and stochastic methods · 890 papers
Keywords: coordinate, proximal, bounds
Approaches in this cluster:
- Adaptive first-order convex methods (243 papers)
Gives convergence rates for adaptive and generalized-smoothness first-order methods on convex problems. - Stochastic gradient last-iterate analysis (226 papers)
Analyzes convergence of SGD and adaptive stepsizes under noise, interpolation, and nonsmooth settings. - Coordinate and Adam convergence analysis (226 papers)
Proves convergence under Lojasiewicz and nonuniform smoothness for coordinate descent, Adam, and steepest descent. - Acceleration and Nesterov dynamics (105 papers)
Studies accelerated gradient methods through geometric, continuous-time, and physical-system views. - Langevin sampling for nonconvex problems (90 papers)
Analyzes Langevin and mirror-Langevin dynamics for sampling and noisy SGD in nonconvex settings.
Most cited and most cited since 2024:
- SGDR: Stochastic Gradient Descent with Warm Restarts (ICLR 2017 · 1,699 citations)
- No Spurious Local Minima in Nonconvex Low Rank Problems: A Unified Geometric Analysis (ICML 2017 · 215 citations)
- PPO-Clip Attains Global Optimality: Towards Deeper Understandings of Clipping (AAAI 2024 · 22 citations)
- Adaptive Proximal Gradient Method for Convex Optimization (NeurIPS 2024 · 13 citations)
Complexity of nonconvex and bilevel problems · 409 papers
Keywords: complexity, bilevel, stationary
Approaches in this cluster:
- Nonconvex minimax and constrained methods (116 papers)
Proposes first-order and gradient-free algorithms for nonsmooth, constrained, and minimax nonconvex problems. - Stochastic convex optimization complexity (105 papers)
Establishes sample and oracle complexity of gradient methods for stochastic convex problems. - Complexity of nonconvex first-order methods (86 papers)
Derives rates for finding stationary points using first-order and inexact-oracle methods. - Penalty and implicit-gradient bilevel methods (75 papers)
Develops first-order and penalty-based algorithms for bilevel optimization with convergence guarantees. - Saddle-point escape algorithms (27 papers)
Designs gradient and zeroth-order methods that escape saddle points to second-order stationary points.
Most cited and most cited since 2024:
- SPIDER: Near-Optimal Non-Convex Optimization via Stochastic Path-Integrated Differential Estimator (NeurIPS 2018 · 243 citations)
- How to Escape Saddle Points Efficiently (ICML 2017 · 219 citations)
- On Partial Optimal Transport: Revising the Infeasibility of Sinkhorn and Efficient Gradient Methods (AAAI 2024 · 5 citations)
- Trust Region Methods for Nonconvex Stochastic Optimization beyond Lipschitz Smoothness (AAAI 2024 · 4 citations)
Variance reduction · 172 papers
Keywords: variance, reduction, reduced
Approaches in this cluster:
- Stochastic variance reduction (102 papers)
Introduces SVRG-style variance-reduced gradient methods with improved rates for nonconvex optimization. - Proximal finite-sum methods (53 papers)
Develops proximal and incremental stochastic methods for nonsmooth finite-sum problems. - Variance-reduced policy gradient (17 papers)
Applies variance reduction to policy gradient and policy evaluation in reinforcement learning.
Most cited and most cited since 2024:
- Stochastic Variance Reduction for Nonconvex Optimization (ICML 2016 · 226 citations)
- Non-convex Finite-Sum Optimization Via SCSG Methods (NeurIPS 2017 · 129 citations)
- Faster Stochastic Variance Reduction Methods for Compositional MiniMax Optimization (AAAI 2024 · 3 citations)
- Universality of AdaGrad Stepsizes for Stochastic Optimization: Inexact Oracle, Acceleration and Variance Reduction (NeurIPS 2024 · 2 citations)
Minimax and games · 169 papers
Keywords: games, zero sum, player
Approaches in this cluster:
- Equilibrium learning in zero-sum games (98 papers)
Proves last-iterate and no-regret convergence of gradient dynamics to Nash and minmax equilibria. - Extragradient for variational inequalities (37 papers)
Develops extragradient and optimistic methods for monotone and nonmonotone variational inequalities. - Gradient descent ascent for minimax (34 papers)
Analyzes local convergence and generalization of gradient descent ascent in GAN-style minimax training.
Most cited and most cited since 2024:
- On Gradient Descent Ascent for Nonconvex-Concave Minimax Problems (ICML 2020 · 183 citations)
- Solving a Class of Non-Convex Min-Max Games Using Iterative First Order Methods (NeurIPS 2019 · 138 citations)
- Approximating Nash Equilibria in Normal-Form Games via Stochastic Optimization (ICLR 2024 · 2 citations)
- Beyond Monotonicity: On the Convergence of Learning Algorithms in Standard Auction Games (AAAI 2025 · 2 citations)
Frank-Wolfe methods · 67 papers
Keywords: frank, wolfe, conditional
Approaches in this cluster:
- Stochastic Frank-Wolfe methods (43 papers)
Develops stochastic, variance-reduced and zeroth-order Frank-Wolfe algorithms with convergence rates. - Projection-free conditional gradient (24 papers)
Designs conditional gradient variants that avoid projections for constrained convex and online optimization.
Most cited and most cited since 2024:
- Variance-Reduced and Projection-Free Stochastic Optimization (ICML 2016 · 74 citations)
- Linear Convergence of a Frank-Wolfe Type Algorithm over Trace-Norm Balls (NeurIPS 2017 · 34 citations)
- Sarah Frank-Wolfe: Methods for Constrained Optimization with Best Rates and Practical Features (ICML 2024 · 2 citations)
- Last-Iterate Convergence for Generalized Frank-Wolfe in Monotone Variational Inequalities (NeurIPS 2024 · 1 citations)
Related topics in Optimization
- Distributed and compressed (388)
- Implicit bias and generalization (1,070)
- Optimizers (1,206)
