ICLR 2026 accepted papers on Optimization
116 of the 5,358 papers accepted at ICLR 2026 (2.2%) are on Optimization, compared with 3.3% at ICLR 2025. They fall into 4 topics.
See ICLR 2026 in the interactive map
Working on something in this area? Describe your idea in scime atlas to see the closest accepted papers by meaning and how crowded the spot has become.
Topics and papers
Implicit bias and generalization · 48 papers
- A Derandomization Framework for Structure Discovery: Applications in Neural Networks and Beyond
- Barriers for Learning in an Evolving World: Mathematical Understanding of Loss of Plasticity
- Closed-form $\ell_r$ norm scaling with data for overparameterized linear regression and diagonal linear networks under $\ell_p$ bias
- Convergence Dynamics of Over-Parameterized Score Matching for a Single Gaussian
- Directional Convergence, Benign Overfitting of Gradient Descent in leaky ReLU two-layer Neural Networks
- Egalitarian Gradient Descent: A Simple Approach to Accelerated Grokking
- Entropic Confinement and Mode Connectivity in Overparameterized Neural Networks
- Generalization Below the Edge of Stability: The Role of Data Geometry
- Hyperbolic Aware Minimization: Implicit Bias for Sparsity
- Implicit Bias and Loss of Plasticity in Matrix Completion: Depth Promotes Low-Rankness
- Interactive Learning of Single-Index Models via Stochastic Gradient Descent
- Intrinsic training dynamics of deep neural networks
and 36 more in the interactive map.
Nonconvex and smooth optimization · 34 papers
- A Schrödinger Eigenfunction Method for Long-Horizon Stochastic Optimal Control
- Beyond Short Steps in Frank-Wolfe Algorithms
- Bilevel Optimization with Lower-Level Uniform Convexity: Theory and Algorithm
- Clipped Gradient Methods for Nonsmooth Convex Optimization under Heavy-Tailed Noise: A Refined Analysis
- Convergence of Muon with Newton-Schulz
- Convex Dominance in Deep Learning I: A Scaling Law of Loss and Learning Rate
- Corner Gradient Descent
- DADA: Dual Averaging with Distance Adaptation
- Decentralized Nonconvex Optimization under Heavy-Tailed Noise: Normalization and Optimal Convergence
- Dual Optimistic Ascent (PI Control) is the Augmented Lagrangian Method in Disguise
- Fast Frank–Wolfe Algorithms with Adaptive Bregman Step-Size for Weakly Convex Functions
- Gradient-Normalized Smoothness for Optimization with Approximate Hessians
and 22 more in the interactive map.
Optimizers · 23 papers
- Cautious Optimizers: Improving Training with One Line of Code (3 citations)
- A Physics-Inspired Optimizer: Velocity Regularized Adam
- Binomial Gradient-Based Meta-Learning for Enhanced Meta-Gradient Estimation
- Differentiable Simulation of Hard Contacts with Soft Gradients for Learning and Control
- Globally aware optimization with resurgence
- High-dimensional limit theorems for SGD: Momentum and Adaptive Step-sizes
- Learning of Population Dynamics: Inverse Optimization Meets JKO Scheme
- MuonBP: Faster Muon via Block-Periodic Orthogonalization
- Taming Score-Based Denoisers in ADMM: A Convergent Plug-and-Play Framework
- The Potential of Second-Order Optimization for LLMs: A Study with Full Gauss-Newton
- Towards Efficient Optimizer Design for LLM via Structured Fisher Approximation with a Low-Rank Extension
- Adaptive gradient descent on Riemannian manifolds and its applications to Gaussian variational inference
and 11 more in the interactive map.
Distributed and compressed · 11 papers
- Communication-Efficient Decentralized Optimization via Double-Communication Symmetric ADMM
- Composite Optimization with Error Feedback: the Dual Averaging Approach
- DeMo: Decoupled Momentum Optimization
- Error Feedback for Muon and Friends
- MT-DAO: Multi-Timescale Distributed Adaptive Optimizers with Local Updates
- Proving the Limited Scalability of Centralized Distributed Optimization via a New Lower Bound Construction
- Birch SGD: A Tree Graph Framework for Local and Asynchronous SGD Methods
- DES-LOC: Desynced Low Communication Adaptive Optimizers for Foundation Models
- LEGACY: A Lightweight Dynamic Gradient Compression Strategy for Distributed Deep Learning
- On The Surprising Effectiveness of a Single Global Merging in Decentralized Learning
- Ringleader ASGD: The First Asynchronous SGD with Optimal Time Complexity under Data Heterogeneity