Nonconvex and smooth optimization: research map
1,448 accepted papers on Nonconvex and smooth optimization in Optimization, from ICML, NeurIPS, ICLR, CVPR and AAAI (2016–2026), grouped into 5 clusters and 18 approaches. The busiest year so far is 2025.
Within Optimization, its share held steady from 36.9% in 2023–24 to 36.0% in 2025–26 (320 → 352 papers at ICML, NeurIPS, CVPR and AAAI, the venues with data for all four years).
Explore Nonconvex and smooth optimization in the interactive map
Working on something in this topic? Describe your idea in scime atlas to see which approach it falls under, the closest papers by meaning and how crowded the spot has become.
Approaches and key papers
coordinate · proximal · bounds · 717 papers
Approaches in this cluster:
- Adaptive first-order convex methods (190 papers)
Gives convergence rates for adaptive and generalized-smoothness first-order methods on convex problems. - Stochastic gradient last-iterate analysis (178 papers)
Analyzes convergence of SGD and adaptive stepsizes under noise, interpolation, and nonsmooth settings. - Coordinate and Adam convergence analysis (188 papers)
Proves convergence under Lojasiewicz and nonuniform smoothness for coordinate descent, Adam, and steepest descent. - Acceleration and Nesterov dynamics (91 papers)
Studies accelerated gradient methods through geometric, continuous-time, and physical-system views. - Langevin sampling for nonconvex problems (70 papers)
Analyzes Langevin and mirror-Langevin dynamics for sampling and noisy SGD in nonconvex settings.
Most cited and most cited since 2024:
- SGDR: Stochastic Gradient Descent with Warm Restarts (ICLR 2017 · 1,699 citations)
- No Spurious Local Minima in Nonconvex Low Rank Problems: A Unified Geometric Analysis (ICML 2017 · 215 citations)
- PPO-Clip Attains Global Optimality: Towards Deeper Understandings of Clipping (AAAI 2024 · 22 citations)
- Adaptive Proximal Gradient Method for Convex Optimization (NeurIPS 2024 · 13 citations)
complexity · bilevel · stationary · 361 papers
Approaches in this cluster:
- Nonconvex minimax and constrained methods (100 papers)
Proposes first-order and gradient-free algorithms for nonsmooth, constrained, and minimax nonconvex problems. - Stochastic convex optimization complexity (94 papers)
Establishes sample and oracle complexity of gradient methods for stochastic convex problems. - Complexity of nonconvex first-order methods (79 papers)
Derives rates for finding stationary points using first-order and inexact-oracle methods. - Penalty and implicit-gradient bilevel methods (65 papers)
Develops first-order and penalty-based algorithms for bilevel optimization with convergence guarantees. - Saddle-point escape algorithms (23 papers)
Designs gradient and zeroth-order methods that escape saddle points to second-order stationary points.
Most cited and most cited since 2024:
- SPIDER: Near-Optimal Non-Convex Optimization via Stochastic Path-Integrated Differential Estimator (NeurIPS 2018 · 243 citations)
- How to Escape Saddle Points Efficiently (ICML 2017 · 219 citations)
- On Partial Optimal Transport: Revising the Infeasibility of Sinkhorn and Efficient Gradient Methods (AAAI 2024 · 5 citations)
- Trust Region Methods for Nonconvex Stochastic Optimization beyond Lipschitz Smoothness (AAAI 2024 · 4 citations)
games · zero sum · player · 154 papers
Approaches in this cluster:
- Equilibrium learning in zero-sum games (88 papers)
Proves last-iterate and no-regret convergence of gradient dynamics to Nash and minmax equilibria. - Extragradient for variational inequalities (35 papers)
Develops extragradient and optimistic methods for monotone and nonmonotone variational inequalities. - Gradient descent ascent for minimax (31 papers)
Analyzes local convergence and generalization of gradient descent ascent in GAN-style minimax training.
Most cited and most cited since 2024:
- On Gradient Descent Ascent for Nonconvex-Concave Minimax Problems (ICML 2020 · 183 citations)
- Solving a Class of Non-Convex Min-Max Games Using Iterative First Order Methods (NeurIPS 2019 · 138 citations)
- Approximating Nash Equilibria in Normal-Form Games via Stochastic Optimization (ICLR 2024 · 2 citations)
- Beyond Monotonicity: On the Convergence of Learning Algorithms in Standard Auction Games (AAAI 2025 · 2 citations)
variance · reduction · reduced · 150 papers
Approaches in this cluster:
- Stochastic variance reduction (86 papers)
Introduces SVRG-style variance-reduced gradient methods with improved rates for nonconvex optimization. - Proximal finite-sum methods (47 papers)
Develops proximal and incremental stochastic methods for nonsmooth finite-sum problems. - Variance-reduced policy gradient (17 papers)
Applies variance reduction to policy gradient and policy evaluation in reinforcement learning.
Most cited and most cited since 2024:
- Stochastic Variance Reduction for Nonconvex Optimization (ICML 2016 · 226 citations)
- Non-convex Finite-Sum Optimization Via SCSG Methods (NeurIPS 2017 · 129 citations)
- Faster Stochastic Variance Reduction Methods for Compositional MiniMax Optimization (AAAI 2024 · 3 citations)
- Universality of AdaGrad Stepsizes for Stochastic Optimization: Inexact Oracle, Acceleration and Variance Reduction (NeurIPS 2024 · 2 citations)
frank · wolfe · conditional · 66 papers
Approaches in this cluster:
- Stochastic Frank-Wolfe methods (43 papers)
Develops stochastic, variance-reduced and zeroth-order Frank-Wolfe algorithms with convergence rates. - Projection-free conditional gradient (23 papers)
Designs conditional gradient variants that avoid projections for constrained convex and online optimization.
Most cited and most cited since 2024:
- Variance-Reduced and Projection-Free Stochastic Optimization (ICML 2016 · 74 citations)
- Linear Convergence of a Frank-Wolfe Type Algorithm over Trace-Norm Balls (NeurIPS 2017 · 34 citations)
- Sarah Frank-Wolfe: Methods for Constrained Optimization with Best Rates and Practical Features (ICML 2024 · 2 citations)
- Last-Iterate Convergence for Generalized Frank-Wolfe in Monotone Variational Inequalities (NeurIPS 2024 · 1 citations)
Related topics in Optimization
- Distributed and compressed (349)
- Implicit bias and generalization (958)
- Optimizers (1,124)