ICLR 2025 accepted papers on Optimization
123 of the 3,703 papers accepted at ICLR 2025 (3.3%) are on Optimization, compared with 3.2% at ICLR 2024. They fall into 4 topics.
See ICLR 2025 in the interactive map
Working on something in this area? Describe your idea in scime atlas to see the closest accepted papers by meaning and how crowded the spot has become.
Topics and papers
Nonconvex and smooth optimization · 51 papers
- Accelerated Over-Relaxation Heavy-Ball Method: Achieving Global Accelerated Convergence with Broad Generalization (1 citation)
- Nonconvex Stochastic Optimization under Heavy-Tailed Noises: Optimal Convergence without Gradient Clipping (1 citation)
- Tuning-Free Bilevel Optimization: New Algorithms and Convergence Analysis (1 citation)
- A Coefficient Makes SVRG Effective
- Adam Exploits $\ell_\infty$-geometry of Loss Landscape via Coordinate-wise Adaptivity
- Adaptive Batch Size for Privately Finding Second-Order Stationary Points
- ADMM for Nonconvex Optimization under Minimal Continuity Assumption
- ADMM for Structured Fractional Minimization
- Boosting Perturbed Gradient Ascent for Last-Iterate Convergence in Games
- Cauchy-Schwarz Regularizers
- Complexity Lower Bounds of Adaptive Gradient Algorithms for Non-convex Stochastic Optimization under Relaxed Smoothness
- CONGO: Compressive Online Gradient Optimization
and 39 more in the interactive map.
Implicit bias and generalization · 36 papers
- Grokking at the Edge of Numerical Stability (1 citation)
- Local Loss Optimization in the Infinite Width: Stable Parameterization of Predictive Coding Networks and Target Propagation (1 citation)
- Loss Landscape of Shallow ReLU-like Neural Networks: Stationary Points, Saddle Escape, and Network Embedding (1 citation)
- Sharpness-Aware Minimization: General Analysis and Improved Rates (1 citation)
- Universal Sharpness Dynamics in Neural Network Training: Fixed Point Analysis, Edge of Stability, and Route to Chaos (1 citation)
- Analysis of Linear Mode Connectivity via Permutation-Based Weight Matching: With Insights into Other Permutation Search Methods
- Analyzing Neural Scaling Laws in Two-Layer Networks with Power-Law Data Spectra
- Convergence and Implicit Bias of Gradient Descent on Continual Linear Classification
- Convex Formulations for Training Two-Layer ReLU Neural Networks
- Deep Weight Factorization: Sparse Learning Through the Lens of Artificial Symmetries
- Do Deep Neural Network Solutions Form a Star Domain?
- Does SGD really happen in tiny subspaces?
and 24 more in the interactive map.
Optimizers · 27 papers
- Adaptive backtracking line search (1 citation)
- MotherNet: Fast Training and Inference via Hyper-Network Transformers (1 citation)
- Optimization by Parallel Quasi-Quantum Annealing with Gradient-Based Sampling (1 citation)
- Revisit Micro-batch Clipping: Adaptive Data Pruning via Gradient Manipulation (1 citation)
- Accelerated training through iterative gradient propagation along the residual path
- AdaFisher: Adaptive Second Order Optimization via Fisher Information
- AdaGrad under Anisotropic Smoothness
- ADAM Optimization with Adaptive Batch Selection
- Adaptive Methods through the Lens of SDEs: Theoretical Insights on the Role of Noise
- Addax: Utilizing Zeroth-Order Gradients to Improve Memory Efficiency and Performance of SGD for Fine-Tuning Language Models
- Debiasing Mini-Batch Quadratics for Applications in Deep Learning
- From Search to Sampling: Generative Models for Robust Algorithmic Recourse
and 15 more in the interactive map.
Distributed and compressed · 9 papers
- Scalable Decentralized Learning with Teleportation (1 citation)
- Adaptive Gradient Clipping for Robust Federated Learning
- Convergence of Distributed Adaptive Optimization with Local Updates
- From Promise to Practice: Realizing High-performance Decentralized Training
- Local Steps Speed Up Local GD for Heterogeneous Distributed Logistic Regression
- Newton Meets Marchenko-Pastur: Massively Parallel Second-Order Optimization with Hessian Sketching and Debiasing
- Towards Faster Decentralized Stochastic Optimization with Communication Compression
- An Asynchronous Bundle Method for Distributed Learning Problems
- Tight Time Complexities in Parallel Stochastic Optimization with Arbitrary Computation Dynamics