NeurIPS 2025 accepted papers on Optimization
186 of the 5,812 papers accepted at NeurIPS 2025 (3.2%) are on Optimization, compared with 4.2% at NeurIPS 2024. They fall into 4 topics.
See NeurIPS 2025 in the interactive map
Working on something in this area? Describe your idea in scime atlas to see the closest accepted papers by meaning and how crowded the spot has become.
Topics and papers
Nonconvex and smooth optimization · 67 papers
- Rewind-to-Delete: Certified Machine Unlearning for Nonconvex Functions (2 citations)
- Adaptive Riemannian ADMM for Nonsmooth Optimization: Optimal Complexity without Smoothing (1 citation)
- Gradient-Variation Online Adaptivity for Accelerated Optimization with Hölder Smoothness (1 citation)
- A Computationally Viable Numerical Gradient-based Technique for Optimal Covering Problems
- A Near-Optimal Algorithm for Decentralized Convex-Concave Finite-Sum Minimax Optimization
- A Regularized Newton Method for Nonconvex Optimization with Global and Local Complexity Guarantees
- A Single-Loop First-Order Algorithm for Linearly Constrained Bilevel Optimization
- A Single-Loop Gradient Algorithm for Pessimistic Bilevel Optimization via Smooth Approximation
- A Unified Analysis of Stochastic Gradient Descent with Arbitrary Data Permutations and Beyond
- Accelerated Distance-adaptive Methods for Hölder Smooth and Convex Optimization
- Acceleration via silver step-size on Riemannian manifolds with applications to Wasserstein space
- Adaptive Algorithms with Sharp Convergence Rates for Stochastic Hierarchical Optimization
and 55 more in the interactive map.
Optimizers · 56 papers
- Small Batch Size Training for Language Models: When Vanilla SGD Works, and Why Gradient Accumulation is Wasteful (4 citations)
- PseuZO: Pseudo-Zeroth-Order Algorithm for Training Deep Neural Networks (3 citations)
- Improving Energy Natural Gradient Descent through Woodbury, Momentum, and Randomization (2 citations)
- Momentum-SAM: Sharpness Aware Minimization without Computational Overhead (2 citations)
- PaZO: Preconditioned Accelerated Zeroth-Order Optimization for Fine-Tuning LLMs (2 citations)
- CAMO: Convergence-Aware Multi-Fidelity Bayesian Optimization (1 citation)
- Enhanced Cyclic Coordinate Descent Methods for Elastic Net Penalized Linear Models (1 citation)
- FSNet: Feasibility-Seeking Neural Network for Constrained Optimization with Guarantees (1 citation)
- PiKE: Adaptive Data Mixing for Large-Scale Multi-Task Learning Under Low Gradient Conflicts (1 citation)
- A Difference-of-Convex Functions Approach to Energy-Based Iterative Reasoning
- A Gradient Guided Diffusion Framework for Chance Constrained Programming
- A Stable Whitening Optimizer for Efficient Neural Network Training
and 44 more in the interactive map.
Implicit bias and generalization · 56 papers
- Better NTK Conditioning: A Free Lunch from (ReLU) Nonlinear Activation in Wide Neural Networks (2 citations)
- Controlling the Flow: Stability and Convergence for Stochastic Gradient Descent with Decaying Regularization (2 citations)
- Asymptotics of SGD in Sequence-Single Index Models and Single-Layer Attention Networks (1 citation)
- Differentiable Sparsity via $D$-Gating: Simple and Versatile Structured Penalization (1 citation)
- Gaussian Approximation and Concentration of Constant Learning-Rate Stochastic Gradient Descent (1 citation)
- Neural Collapse under Gradient Flow on Shallow ReLU Networks for Orthogonally Separable Data (1 citation)
- Non-Singularity of the Gradient Descent Map for Neural Networks with Piecewise Analytic Activations (1 citation)
- A Closer Look at NTK Alignment: Linking Phase Transitions in Deep Image Regression
- A Theoretical Framework for Grokking: Interpolation followed by Riemannian Norm Minimisation
- A Unified Stability Analysis of SAM vs SGD: Role of Data Coherence and Emergence of Simplicity Bias
- Any-stepsize Gradient Descent for Separable Data under Fenchel–Young Losses
- Ascent Fails to Forget
and 44 more in the interactive map.
Distributed and compressed · 7 papers
- Sketched Adaptive Distributed Deep Learning: A Sharp Convergence Analysis (1 citation)
- Unveiling the Power of Multiple Gossip Steps: A Stability-Based Generalization Analysis in Decentralized Training (1 citation)
- ACCO: Accumulate While You Communicate for Communication-Overlapped Sharded LLM Training
- Approximate Gradient Coding for Distributed Learning with Heterogeneous Stragglers
- Revisiting Consensus Error: A Fine-grained Analysis of Local SGD under Second-order Data Heterogeneity
- Tight analyses of first-order methods with error feedback
- Understanding outer learning rates in Local SGD