ICLR 2022 accepted papers on Optimization
65 of the 1,095 papers accepted at ICLR 2022 (5.9%) are on Optimization, compared with 6.0% at ICLR 2021. They fall into 4 topics.
See ICLR 2022 in the interactive map
Submitting to ICLR? Checklist →
Working on something in this area? Describe your idea in scime atlas to see the closest papers by meaning and how crowded the spot has become.
Topics and papers
Implicit bias and generalization · 24 papers
- Efficient Sharpness-aware Minimization for Improved Training of Neural Networks (25 citations)
- Assessing Generalization of SGD via Disagreement (19 citations)
- Surrogate Gap Minimization Improves Sharpness-Aware Training (13 citations)
- An Unconstrained Layer-Peeled Perspective on Neural Collapse (11 citations)
- Demystifying Batch Normalization in ReLU Networks: Equivalent Convex Optimization Models and Implicit Regularization (9 citations)
- Stochastic Training is Not Necessary for Generalization (6 citations)
- Strength of Minibatch Noise in SGD (6 citations)
- A global convergence theory for deep ReLU implicit networks via over-parameterization (5 citations)
- Large Learning Rate Tames Homogeneity: Convergence and Balancing Effect (4 citations)
- What Happens after SGD Reaches Zero Loss? --A Mathematical Framework (3 citations)
- The Convex Geometry of Backpropagation: Neural Network Gradient Flows Converge to Extreme Points of the Dual Convex Program (2 citations)
- Implicit Bias of MSE Gradient Optimization in Underparameterized Neural Networks (1 citation)
and 12 more in the interactive map.
Optimizers · 22 papers
- Continuous-Time Meta-Learning with Forward Mode Differentiation (6 citations)
- Interacting Contour Stochastic Gradient Langevin Dynamics (4 citations)
- Optimizing Neural Networks with Gradient Lexicase Selection (4 citations)
- Doubly Adaptive Scaled Algorithm for Machine Learning Using Second-Order Information (3 citations)
- Online Hyperparameter Meta-Learning with Hypergradient Distillation (3 citations)
- Amortized Implicit Differentiation for Stochastic Bilevel Optimization (2 citations)
- EigenGame Unloaded: When playing games is better than optimizing (2 citations)
- FastSHAP: Real-Time Shapley Value Estimation (2 citations)
- Half-Inverse Gradients for Physical Deep Learning (2 citations)
- Understanding approximate and unrolled dictionary learning for pattern recovery (2 citations)
- Domain Adversarial Training: A Game Perspective (1 citation)
- Memory Augmented Optimizers for Deep Learning (1 citation)
and 10 more in the interactive map.
Nonconvex and smooth optimization · 16 papers
- Shuffle Private Stochastic Convex Optimization (6 citations)
- Escaping limit cycles: Global convergence for constrained nonconvex-nonconcave minimax problems (3 citations)
- Hidden Convexity of Wasserstein GANs: Interpretable Generative Models with Closed-Form Solutions (3 citations)
- High Probability Bounds for a Class of Nonconvex Algorithms with AdaGrad Stepsize (3 citations)
- On the Convergence of mSGD and AdaGrad for Stochastic Optimization (2 citations)
- Provably convergent quasistatic dynamics for mean-field two-player zero-sum games (2 citations)
- Bregman Gradient Policy Optimization (1 citation)
- Minibatch vs Local SGD with Shuffling: Tight Convergence Bounds and Beyond (1 citation)
- Minimax Optimization with Smooth Algorithmic Adversaries (1 citation)
- Permutation-Based SGD: Is Random Optimal? (1 citation)
- GDA-AM: ON THE EFFECTIVENESS OF SOLVING MIN-IMAX OPTIMIZATION VIA ANDERSON MIXING
- Training Structured Neural Networks Through Manifold Identification and Variance Reduction
and 4 more in the interactive map.
