ICLR 2023 accepted papers on Optimization
76 of the 1,575 papers accepted at ICLR 2023 (4.8%) are on Optimization, compared with 5.9% at ICLR 2022. They fall into 4 topics.
See ICLR 2023 in the interactive map
Working on something in this area? Describe your idea in scime atlas to see the closest accepted papers by meaning and how crowded the spot has become.
Topics and papers
Implicit bias and generalization · 27 papers
- Git Re-Basin: Merging Models modulo Permutation Symmetries (33 citations)
- Neural Networks Efficiently Learn Low-Dimensional Representations with SGD (6 citations)
- Understanding the Generalization of Adam in Learning Neural Networks with Proper Regularization (6 citations)
- Implicit Bias in Leaky ReLU Networks Trained on High-Dimensional Data (5 citations)
- A Kernel Perspective of Skip Connections in Convolutional Networks (3 citations)
- Self-Stabilization: The Implicit Bias of Gradient Descent at the Edge of Stability (3 citations)
- The Implicit Bias of Minima Stability in Multivariate Shallow ReLU Networks (3 citations)
- Symmetries, Flat Minima, and the Conserved Quantities of Gradient Flow (2 citations)
- Excess Risk of Two-Layer ReLU Neural Networks in Teacher-Student Settings and its Superiority to Kernel Methods (1 citation)
- Implicit regularization in Heavy-ball momentum accelerated stochastic gradient descent (1 citation)
- Parallel Deep Neural Networks Have Zero Duality Gap (1 citation)
- Understanding Edge-of-Stability Training Dynamics with a Minimalist Example (1 citation)
and 15 more in the interactive map.
Optimizers · 23 papers
- FairGBM: Gradient Boosting with Fairness Constraints (6 citations)
- Concept Gradient: Concept-based Interpretation Without Linear Assumption (5 citations)
- Noise Is Not the Main Factor Behind the Gap Between Sgd and Adam on Transformers, But Sign Descent Might Be (5 citations)
- Alternating Differentiation for Optimization Layers (3 citations)
- Learning Iterative Neural Optimizers for Image Steganography (3 citations)
- An Adaptive Policy to Employ Sharpness-Aware Minimization (2 citations)
- Mitigating Gradient Bias in Multi-objective Learning: A Provably Convergent Approach (2 citations)
- Sampling with Mollified Interaction Energy Descent (2 citations)
- D4FT: A Deep Learning Approach to Kohn-Sham Density Functional Theory (1 citation)
- Predictor-corrector algorithms for stochastic optimization under gradual distribution shift (1 citation)
- ISAAC Newton: Input-based Approximate Curvature for Newton's Method
- Learning Proximal Operators to Discover Multiple Optima
and 11 more in the interactive map.
Nonconvex and smooth optimization · 19 papers
- A Unified Approach to Reinforcement Learning, Quantal Response Equilibria, and Two-Player Zero-Sum Games (6 citations)
- Accelerated Single-Call Methods for Constrained Min-Max Optimization (3 citations)
- Beyond Lipschitz: Sharp Generalization and Excess Risk Bounds for Full-Batch GD (3 citations)
- The Power of Regularization in Solving Extensive-Form Games (2 citations)
- Continuized Acceleration for Quasar Convex Functions in Non-Convex Optimization (1 citation)
- RandProx: Primal-Dual Optimization Algorithms with Randomized Proximal Updates (1 citation)
- Solving stochastic weak Minty variational inequalities without increasing batch size (1 citation)
- Can We Find Nash Equilibria at a Linear Rate in Markov Games?
- Linear Convergence of Natural Policy Gradient Methods with Log-Linear Policies
- On Accelerated Perceptrons and Beyond
- On the Convergence of AdaGrad(Norm) on $\mathbb{R}^d$: Beyond Convexity, Non-Asymptotic Rate and Acceleration
- Projective Proximal Gradient Descent for Nonconvex Nonsmooth Optimization: Fast Convergence Without Kurdyka-Lojasiewicz (KL) Property
and 7 more in the interactive map.
Distributed and compressed · 7 papers
- Asynchronous Distributed Bilevel Optimization (6 citations)
- Variance Reduction is an Antidote to Byzantines: Better Rates, Weaker Assumptions and Communication Compression as a Cherry on the Top (5 citations)
- Maximizing Communication Efficiency for Large-scale Training via 0/1 Adam (4 citations)
- EPISODE: Episodic Gradient Clipping with Periodic Resampled Corrections for Federated Learning with Heterogeneous Data (2 citations)
- Distributed Extra-gradient with Optimal Complexity and Communication Guarantees
- Sequential Gradient Coding For Straggler Mitigation
- DASHA: Distributed Nonconvex Optimization with Communication Compression and Optimal Oracle Complexity