ICML 2026 accepted papers on Optimization
184 of the 6,341 papers accepted at ICML 2026 (2.9%) are on Optimization, compared with 3.8% at ICML 2025. They fall into 4 topics.
See ICML 2026 in the interactive map
Working on something in this area? Describe your idea in scime atlas to see the closest accepted papers by meaning and how crowded the spot has become.
Topics and papers
Optimizers · 72 papers
- Accurate Large-sample Uncertainty Quantification using Stochastic Gradient Markov Chain Monte Carlo
- Adaptive Batch Sizes Using Non-Euclidean Gradient Noise Scales for Stochastic Sign and Spectral Descent
- Adaptive Momentum and Nonlinear Damping for Neural Network Training
- Adaptive Preconditioners Trigger Loss Spikes in Adam
- Adaptive Sharpness-Aware Minimization with a Polyak-type Step size: A Theory-Grounded Scheduler
- Clipping Bottleneck: Stabilizing RLVR via Stochastic Recovery of Near-Boundary Signals
- Decoupling Variance and Scale-Invariant Updates in Adaptive Gradient Descent for Unified Vector and Matrix Optimization
- Delving into Muon and Beyond: Deep Analysis and Extensions
- Enhancing LLM Training via Spectral Clipping
- Flatness-Aware Stochastic Gradient Langevin Dynamics
- FOAM: Blocked State Folding for Memory-Efficient LLM Training
- Generalizing Stochastic Smoothing for Differentiation and Gradient Estimation
and 60 more in the interactive map.
Nonconvex and smooth optimization · 63 papers
- Gradient Testing and Estimation by Comparisons (1 citation)
- A Fully First-Order Layer for Differentiable Optimization
- A Short and Unified Convergence Analysis of the SAG, SAGA, and IAG Algorithms
- A Tale of Two Problems: Multi-Task Bilevel Learning Meets Equality Constrained Multi-Objective Optimization
- Accelerated and Stable Convergence with Anchored Generalized Optimistic Method
- Accelerated Multiple Wasserstein Gradient Flows for Multi-objective Distributional Optimization
- Accelerating Langevin Monte Carlo via Efficient Stochastic Runge-Kutta Methods beyond Log-Concavity
- All ERMs Can Fail in Stochastic Convex Optimization Lower Bounds in Linear Dimension
- Asymmetric Perturbation in Solving Bilinear Saddle-Point Optimization
- Attention's forward pass and Frank-Wolfe
- Bregman meets Lévy: Stochastic Mirror Descent with Heavy-Tailed Noise in Continuous and Discrete Time
- Can Adaptive Gradient Methods Converge under Heavy-Tailed Noise? A Case Study of AdaGrad
and 51 more in the interactive map.
Implicit bias and generalization · 37 papers
- A Fourier perspective on the learning dynamics of neural networks: from sample complexities to mechanistic insights
- Balancing Learning Rates Across Layers: Exact Two-Step Dynamics and Optimal Scaling in Linear Neural Networks
- Conflicting Biases at the Edge of Stability: Norm versus Sharpness Regularization
- Conservation Laws for Modern Neural Architectures
- Convex Basins in Single-Index Model Loss Landscapes: Applications to Robust Recovery under Strong Adversarial Corruption
- Data Reconstruction: Identifiability and Optimization with Sample Splitting
- Dichotomy of Feature Learning and Unlearning: Fast-Slow Analysis on Neural Networks with Stochastic Gradient Descent
- Dynamics and Representation Structure of Local Approximations to Gradient-Based Learning in Linear Recurrent Neural Networks
- Fix the Loss, Not the Radius: Rethinking the Adversarial Perturbation of Sharpness-Aware Minimization
- Flat Minima and Generalization: Insights from Stochastic Convex Optimization
- Flatland: The Adventures of Gradient Descent with Large Step Sizes
- From Flat Facts to Sharp Hallucinations: Detecting Stubborn Errors via Gradient Sensitivity
and 25 more in the interactive map.
Distributed and compressed · 12 papers
- A Tight Theory of Error Feedback Algorithms in Distributed Optimization
- Clipping Makes Distributed and Federated Asynchronous SGD Robust to Stragglers
- Convergence Analysis of the Lion Optimizer in Centralized and Distributed Settings
- Distributed Direct Preference Optimization
- FAB: A First-Order AB-based Gradient Algorithm for Distributed Bilevel Optimization over Time-Varying Directed Graphs
- Improved Convergence Analysis of Topology Dependence in Decentralized SGD
- Improving the Robustness-Utility Trade-off in Decentralized Learning over Sparse Networks
- On the Interaction of Batch Noise, Adaptivity, and Compression, under $(L_0,L_1)$-Smoothness: An SDE Approach
- Accelerated Dual Method for Distributed Optimization: An Inexact-Gradient View of Local Updates
- Distributed Stochastic $K$-Level Optimization Over Networks
- Identifying and Mitigating Errors in Gradient Aggregation of Distributed Data Parallel Training
- On the Convergence of Decentralized Stochastic Minimax Optimization Algorithm with Compressed Communication