NeurIPS 2026 accepted papers on Optimization
164 of the 6,080 papers accepted at NeurIPS 2026 (2.7%) are on Optimization, compared with 3.2% at NeurIPS 2025. They fall into 4 topics.
See NeurIPS 2026 in the interactive map
Working on something in this area? Describe your idea in scime atlas to see the closest accepted papers by meaning and how crowded the spot has become.
Topics and papers
Nonconvex and smooth optimization · 64 papers
- Adam under Generalized Smoothness with Second-Moment-Type Stochastic Gradients
- Adaptively Incorporating Directional Hints into Zeroth-Order Optimization
- Almost Sure Convergence Rates of Stochastic Approximation and Reinforcement Learning via a Poisson-Moreau Drift
- BROS: Bias-Corrected Randomized Subspaces for Memory-Efficient Single-Loop Bilevel Optimization
- Can Entry-Wise Clipping Give Spectral Control of Stochastic Gradients?
- Complexity of Classical Acceleration for $\ell_1$-Regularized PageRank
- Constrained Stochastic Spectral Preconditioning Converges for Nonconvex Objectives
- Direct Acceleration of Stochastic Root-Finding Without Variance Reduction and Regularization
- EMA-Nesterov: Stabilizing Nesterov's Lookahead for Accelerated Deep Learning Optimization
- Finding Simple Proofs for First-Order Optimization
- Finite-Sample Performance of Gradient Descent in Logistic Regression with Gaussian Design
- Frank-Wolfe Beyond 1/t Convergence
and 52 more in the interactive map.
Implicit bias and generalization · 46 papers
- Asymmetric Scaling Laws from Sparse Features
- Attention Sinks Induce Gradient Sinks: Massive Activations as Gradient Regulators in Transformers
- Convergence Analysis of Newton's Method for Neural Networks in the Overparameterized Limit
- Deep Learning as Neural Low-Degree Filtering: A Spectral Theory of Hierarchical Feature Learning
- Diagonalizing the Softmax: Hadamard Initialization for Tractable Cross-Entropy Dynamics
- Drifting Fields are not Conservative
- Estimating Implicit Regularization in Deep Learning
- Global convergence of adjoint-optimized neural PDEs
- High-Dimensional Learning Dynamics of Attention-Indexed Models
- High-dimensional Limit of SGD for Diagonal Linear Networks
- Implicit Bias of Mirror Flow in Homogeneous Neural Networks: Sparse and Dense Feature Learning
- Minimax Optimal Early-Stopped Gradient Descent for Gaussian Mixture Classification
and 34 more in the interactive map.
Optimizers · 40 papers
- Jacobian Descent for Multi-Objective Optimization (1 citation)
- A Differentiable Interior-Point Method in Single Precision
- AMUSE: Anytime Muon with Stable Gradient Evaluation
- Delve into the Applicability of Advanced Optimizers for Multi-Task Learning
- Denoise First, Orthogonalize Later: Understanding Momentum in Muon via Spectral Filtering
- EM-NeSy: Expectation Maximization for Neurosymbolic Learning
- Factor Augmented High-Dimensional SGD
- From SGD to Muon: Adaptive Optimization via Schatten-p Norms
- Layerwise LQR for Geometry-Aware Optimization of Deep Networks
- LOTION: Smoothing the Optimization Landscape for Quantized Training
- MetaKE: Meta-Learning for Knowledge Editing Toward a Better Accuracy-Editability Trade-off
- Perfect Parallelization in Mini-Batch SGD with Classical Momentum Acceleration
and 28 more in the interactive map.
Distributed and compressed · 14 papers
- Adversary-Robust Learning from Fully Asynchronous Directional Derivative Estimates
- Do We Need Asynchronous SGD? On the Near-Optimality of Synchronous Methods
- Faster Rates For Federated Variational Inequalities
- From Inexact Gradients to Byzantine Robustness: Acceleration and Optimization under Similarity
- LOSCAR-SGD: Local SGD with Communication-Computation Overlap and Delay-Corrected Sparse Model Averaging
- Rescaled Asynchronous SGD: Optimal Distributed Optimization under Data and System Heterogeneity
- RESIST: Resilient Decentralized Learning Using Consensus Gradient Descent
- Scalable Distributed Stochastic Optimization via Bidirectional Compression: Beyond Pessimistic Limits
- Stochastic Matching via Local Sparsification
- Clapping: Removing Per-sample Storage for Pipeline Parallel Learning with Communication Compression
- DPA: Decentralized Primal Averaging with Quasi-Global Momentum for Highly Heterogeneous Data
- Federated Distributionally Robust Neural Combinatorial Optimization with Convergence Guarantees
and 2 more in the interactive map.