ICML 2025 accepted papers on Optimization
127 of the 3,342 papers accepted at ICML 2025 (3.8%) are on Optimization, compared with 5.6% at ICML 2024. They fall into 4 topics.
See ICML 2025 in the interactive map
Working on something in this area? Describe your idea in scime atlas to see the closest accepted papers by meaning and how crowded the spot has become.
Topics and papers
Nonconvex and smooth optimization · 45 papers
- A Near-Optimal Single-Loop Stochastic Algorithm for Convex Finite-Sum Coupled Compositional Optimization
- A Parameter-Free and Near-Optimal Zeroth-Order Algorithm for Stochastic Convex Optimization
- Achieving Linear Speedup and Near-Optimal Complexity for Decentralized Optimization over Row-stochastic Networks
- Active Learning of Deep Neural Networks via Gradient-Free Cutting Planes
- Armijo Line-search Can Make (Stochastic) Gradient Descent Provably Faster
- Clipping Improves Adam-Norm and AdaGrad-Norm when the Noise Is Heavy-Tailed
- Constrained Online Convex Optimization with Polyak Feasibility Steps
- Continuous-Time Analysis of Heavy Ball Momentum in Min-Max Games
- Convergence of Policy Mirror Descent Beyond Compatible Function Approximation
- General framework for online-to-nonconvex conversion: Schedule-free SGD is also effective for nonconvex optimization
- Generalization of noisy SGD in unbounded non-convex settings
- Global Optimization with a Power-Transformed Objective and Gaussian Smoothing
and 33 more in the interactive map.
Optimizers · 42 papers
- FRUGAL: Memory-Efficient Optimization by Reducing State Overhead for Scalable Training (1 citation)
- A Comprehensive Framework for Analyzing the Convergence of Adam: Bridging the Gap with SGD
- A Generalization Result for Convergence in Learning-to-Optimize
- Accelerating PDE-Constrained Optimization by the Derivative of Neural Operators
- Aligned Multi Objective Optimization
- Automatic Differentiation of Optimization Algorithms with Time-Varying Updates
- BSO: Binary Spiking Online Optimization Algorithm
- Differentiable Quadratic Optimization For the Maximum Independent Set Problem
- Efficient Curvature-Aware Hypergradient Approximation for Bilevel Optimization
- Efficient Optimization with Orthogonality Constraint: a Randomized Riemannian Submanifold Method
- EnsLoss: Stochastic Calibrated Loss Ensembles for Preventing Overfitting in Classification
- Exact risk curves of signSGD in High-Dimensions: quantifying preconditioning and noise-compression effects
and 30 more in the interactive map.
Implicit bias and generalization · 26 papers
- The Butterfly Effect: Neural Network Training Trajectories Are Highly Sensitive to Initial Conditions (1 citation)
- Benefits of Early Stopping in Gradient Descent for Overparameterized Logistic Regression
- Deep Linear Network Training Dynamics from Random Initialization: Data, Width, Depth, and Hyperparameter Transfer
- Emergence in non-neural models: grokking modular arithmetic via average gradient outer product
- Feature learning from non-Gaussian inputs: the case of Independent Component Analysis in high dimensions
- Grokking at the Edge of Linear Separability
- Implicit Bias of Gradient Descent for Non-Homogeneous Deep Networks
- Implicit Regularization for Tubal Tensor Factorizations via Gradient Descent
- In-Context Deep Learning via Transformer Models
- Learning Curves of Stochastic Gradient Descent in Kernel Regression
- Mixture of Experts Provably Detect and Learn the Latent Cluster Structure in Gradient-Based Learning
- Neural Collapse Beyond the Unconstrained Features Model: Landscape, Dynamics, and Generalization in the Mean-Field Regime
and 14 more in the interactive map.
Distributed and compressed · 14 papers
- Constant Stepsize Local GD for Logistic Regression: Acceleration by Instability
- Decoupled SGDA for Games with Intermittent Strategy Communication
- Distributed Event-Based Learning via ADMM
- Distributed Retraction-Free and Communication-Efficient Optimization on the Stiefel Manifold
- Efficient Distributed Optimization under Heavy-Tailed Noise
- Exploiting Similarity for Computation and Communication-Efficient Decentralized Optimization
- HALoS: Hierarchical Asynchronous Local SGD over Slow Networks for Geo-Distributed Large Language Model Training
- Riemannian Diffusion Adaptation for Distributed Optimization on Manifolds
- Ringmaster ASGD: The First Asynchronous SGD with Optimal Time Complexity
- Byzantine-Resilient Federated Alternating Gradient Descent and Minimization for Partly-Decoupled Low Rank Matrix Learning
- Distributed Parallel Gradient Stacking(DPGS): Solving Whole Slide Image Stacking Challenge in Multi-Instance Learning
- Enhancing Parallelism in Decentralized Stochastic Convex Optimization
and 2 more in the interactive map.