ICML 2024 accepted papers on Optimization
146 of the 2,610 papers accepted at ICML 2024 (5.6%) are on Optimization, compared with 7.2% at ICML 2023. They fall into 4 topics.
See ICML 2024 in the interactive map
Working on something in this area? Describe your idea in scime atlas to see the closest accepted papers by meaning and how crowded the spot has become.
Topics and papers
Nonconvex and smooth optimization · 64 papers
- Accelerating Convergence of Score-Based Diffusion Models, Provably (5 citations)
- Sarah Frank-Wolfe: Methods for Constrained Optimization with Best Rates and Practical Features (2 citations)
- A connection between Tempering and Entropic Mirror Descent (1 citation)
- Barrier Algorithms for Constrained Non-Convex Optimization (1 citation)
- Compact Optimality Verification for Optimization Proxies (1 citation)
- Convex and Bilevel Optimization for Neural-Symbolic Inference and Learning (1 citation)
- Random Scaling and Momentum for Non-smooth Non-convex Optimization (1 citation)
- Tuning-Free Stochastic Optimization (1 citation)
- Universal Gradient Methods for Stochastic Convex Optimization (1 citation)
- A Nearly Optimal Single Loop Algorithm for Stochastic Bilevel Optimization under Unbounded Smoothness
- A Universal Transfer Theorem for Convex Optimization Algorithms Using Inexact First-order Oracles
- Accelerated Algorithms for Constrained Nonconvex-Nonconcave Min-Max Optimization and Comonotone Inclusion
and 52 more in the interactive map.
Optimizers · 38 papers
- Interpreting and Improving Diffusion Models from an Optimization Perspective (4 citations)
- Prodigy: An Expeditiously Adaptive Parameter-Free Learner (3 citations)
- Constrained Exploration via Reflected Replica Exchange Stochastic Gradient Langevin Dynamics (2 citations)
- Differentiability and Optimization of Multiparameter Persistent Homology (2 citations)
- Understanding Adam Optimizer via Online Learning of Updates: Adam is FTRL in Disguise (2 citations)
- Online Learning and Information Exponents: The Importance of Batch size & Time/Complexity Tradeoffs (1 citation)
- Plug-and-Play image restoration with Stochastic deNOising REgularization (1 citation)
- Revisiting Scalable Hessian Diagonal Approximations for Applications in Reinforcement Learning (1 citation)
- Scaling Exponents Across Parameterizations and Optimizers (1 citation)
- Adaptive Conformal Inference by Betting
- Can We Remove the Square-Root in Adaptive Gradient Methods? A Second-Order Perspective
- Careful with that Scalpel: Improving Gradient Surgery with an EMA
and 26 more in the interactive map.
Implicit bias and generalization · 34 papers
- A Theory of Non-Linear Feature Learning with One Gradient Step in Two-Layer Neural Networks (2 citations)
- Asymptotics of feature learning in two-layer networks after one gradient-step (2 citations)
- Improved Stability and Generalization Guarantees of the Decentralized SGD Algorithm (2 citations)
- On the Origins of Linear Representations in Large Language Models (2 citations)
- Sliding Down the Stairs: How Correlated Latent Variables Accelerate Learning with Neural Networks (2 citations)
- The Benefits of Reusing Batches for Gradient Descent in Two-Layer Networks: Breaking the Curse of Information and Leap Exponents (2 citations)
- Activation-Descent Regularization for Input Optimization of ReLU Networks (1 citation)
- Beyond Implicit Bias: The Insignificance of SGD Noise in Online Learning (1 citation)
- Why Do You Grok? A Theoretical Analysis on Grokking Modular Addition (1 citation)
- A Universal Class of Sharpness-Aware Minimization Algorithms
- Achieving Margin Maximization Exponentially Fast via Progressive Norm Rescaling
- Bias of Stochastic Gradient Descent or the Architecture: Disentangling the Effects of Overparameterization of Neural Networks
and 22 more in the interactive map.
Distributed and compressed · 10 papers
- High-Probability Convergence for Composite and Distributed Stochastic Minimization and Variational Inequalities with Heavy-Tailed Noise (3 citations)
- Distributed Bilevel Optimization with Communication Compression
- Dynamic Byzantine-Robust Learning: Adapting to Switching Byzantine Workers
- Embarrassingly Parallel GFlowNets
- Fault Tolerant ML: Efficient Meta-Aggregation and Synchronous Training
- LASER: Linear Compression in Wireless Distributed Optimization
- A Doubly Recursive Stochastic Compositional Gradient Descent Method for Federated Multi-Level Compositional Optimization
- Double Stochasticity Gazes Faster: Snap-Shot Decentralized Stochastic Gradient Tracking Methods
- Locally Differentially Private Decentralized Stochastic Bilevel Optimization with Guaranteed Convergence Accuracy
- SILVER: Single-loop variance reduction and application to federated learning