ICML 2022 accepted papers on Optimization
106 of the 1,233 papers accepted at ICML 2022 (8.6%) are on Optimization, compared with 8.4% at ICML 2021. They fall into 4 topics.
See ICML 2022 in the interactive map
Submitting to ICML? Checklist →
Working on something in this area? Describe your idea in scime atlas to see the closest papers by meaning and how crowded the spot has become.
Topics and papers
Nonconvex and smooth optimization · 43 papers
- The power of first-order smooth optimization for black-box non-smooth problems (11 citations)
- ProxSkip: Yes! Local Gradient Steps Provably Lead to Communication Acceleration! Finally! (10 citations)
- FedNest: Federated Bilevel, Minimax, and Compositional Optimization (7 citations)
- On Last-Iterate Convergence Beyond Zero-Sum Games (6 citations)
- When AUC meets DRO: Optimizing Partial AUC for Deep Learning with Non-Convex Convergence Guarantee (6 citations)
- Accelerated, Optimal and Parallel: Some results on model-based stochastic optimization (4 citations)
- RECAPP: Crafting a More Efficient Catalyst for Convex Optimization (4 citations)
- A Single-Loop Gradient Descent and Perturbed Ascent Algorithm for Nonconvex Functional Constrained Optimization (3 citations)
- Hessian-Free High-Resolution Nesterov Acceleration For Sampling (3 citations)
- Stochastic Continuous Submodular Maximization: Boosting via Non-oblivious Function (3 citations)
- Convergence Rates of Non-Convex Stochastic Gradient Descent Under a Generic Lojasiewicz Condition and Local Smoothness (2 citations)
- Finite-Sum Coupled Compositional Stochastic Optimization: Theory and Applications (2 citations)
and 31 more in the interactive map.
Optimizers · 28 papers
- Adaptive Inertia: Disentangling the Effects of Adaptive Learning Rate and Momentum (13 citations)
- Sharp-MAML: Sharpness-Aware Model-Agnostic Meta Learning (9 citations)
- Variational Wasserstein gradient flow (7 citations)
- Adaptive Second Order Coresets for Data-efficient Machine Learning (5 citations)
- Do Differentiable Simulators Give Better Policy Gradients? (5 citations)
- Improve Single-Point Zeroth-Order Optimization Using High-Pass and Low-Pass Filters (4 citations)
- Injecting Logical Constraints into Neural Networks via Straight-Through Estimators (4 citations)
- Large-scale Stochastic Optimization of NDCG Surrogates for Deep Learning with Provable Convergence (4 citations)
- A Convergence Theory for SVGD in the Population Limit under Talagrand’s Inequality T1 (3 citations)
- Analysis of Stochastic Processes through Replay Buffers (3 citations)
- Biased Gradient Estimate with Drastic Variance Reduction for Meta Reinforcement Learning (3 citations)
- On Implicit Bias in Overparameterized Bilevel Optimization (3 citations)
and 16 more in the interactive map.
Implicit bias and generalization · 23 papers
- Penalizing Gradient Norm for Efficiently Improving Generalization in Deep Learning (15 citations)
- Towards Understanding Sharpness-Aware Minimization (11 citations)
- Deep Networks on Toroids: Removing Symmetries Reveals the Structure of Flat Regions in the Landscape Geometry (9 citations)
- Understanding the unstable convergence of gradient descent (8 citations)
- The dynamics of representation learning in shallow, non-linear autoencoders (7 citations)
- Topology-aware Generalization of Decentralized SGD (7 citations)
- Towards understanding how momentum improves generalization in deep learning (7 citations)
- Multi-scale Feature Learning Dynamics: Insights for Double Descent (6 citations)
- Understanding Gradient Descent on the Edge of Stability in Deep Learning (4 citations)
- Neural Network Weights Do Not Converge to Stationary Points: An Invariant Measure Perspective (3 citations)
- An Initial Alignment between Neural Network and Target is Needed for Gradient Descent to Learn (2 citations)
- Anticorrelated Noise Injection for Improved Generalization (2 citations)
and 11 more in the interactive map.
Distributed and compressed · 12 papers
- Byzantine Machine Learning Made Easy By Resilient Averaging of Momentums (8 citations)
- 3PC: Three Point Compressors for Communication-Efficient Distributed Training and a Better Theory for Lazy Aggregation (6 citations)
- Federated Minimax Optimization: Improved Convergence Analyses and Algorithms (6 citations)
- Proximal and Federated Random Reshuffling (4 citations)
- Detached Error Feedback for Distributed SGD with Random Sparsification (2 citations)
- Tackling Data Heterogeneity: A New Unified Framework for Decentralized SGD with Sample-induced Topology (2 citations)
- Correlated Quantization for Distributed Mean Estimation and Optimization (1 citation)
- Lightweight Projective Derivative Codes for Compressed Asynchronous Gradient Descent
- A Stochastic Multi-Rate Control Framework For Modeling Distributed Optimization Algorithms
- Communication-efficient Distributed Learning for Large Batch Optimization
- DRAGONN: Distributed Randomized Approximate Gradients of Neural Networks
- On the Convergence of Local Stochastic Compositional Gradient Descent with Momentum
