ICML 2023 accepted papers on Optimization

Nonconvex and smooth optimization · 58 papers

• On the Convergence Rates of Policy Gradient Methods (12 citations)
• Optimal Stochastic Non-smooth Non-convex Optimization through Online-to-Non-convex Conversion (6 citations)
• Accelerated Primal-Dual Methods for Convex-Strongly-Concave Saddle Point Problems (5 citations)
• High-Probability Bounds for Stochastic Optimization and Variational Inequalities: the Case of Unbounded Variance (5 citations)
• Learning-Rate-Free Learning by D-Adaptation (5 citations)
• Revisiting Gradient Clipping: Stochastic bias and tight convergence guarantees (4 citations)
• On Penalty-based Bilevel Gradient Descent Method (3 citations)
• Accelerated Infeasibility Detection of Constrained Optimization and Fixed-Point Iterations (2 citations)
• Averaged Method of Multipliers for Bi-Level Optimization without Lower-Level Strong Convexity (2 citations)
• Complexity of Block Coordinate Descent with Proximal Regularization and Applications to Wasserstein CP-dictionary Learning (2 citations)
• Faster Gradient-Free Algorithms for Nonsmooth Nonconvex Stochastic Optimization (2 citations)
• Faster Rates of Convergence to Stationary Points in Differentially Private Optimization (2 citations)

and 46 more in the interactive map.

Implicit bias and generalization · 38 papers

• Transformers Learn In-Context by Gradient Descent (88 citations)
• Dropout Reduces Underfitting (15 citations)
• The Power of Preconditioning in Overparameterized Low-Rank Matrix Sensing (10 citations)
• Stochastic Gradient Descent-Induced Drift of Representation in a Two-Layer Neural Network (7 citations)
• Special Properties of Gradient Descent with Large Learning Rates (6 citations)
• Neural networks trained with SGD learn distributions of increasing complexity (5 citations)
• Global Convergence of Sub-gradient Method for Robust Matrix Recovery: Small Initialization, Noisy Measurements, and Over-parameterization (4 citations)
• SGD with Large Step Sizes Learns Sparse Features (4 citations)
• A Theoretical Analysis of the Learning Dynamics under Class Imbalance (3 citations)
• Analyzing Convergence in Quantum Neural Networks: Deviations from Neural Tangent Kernels (3 citations)
• Algorithmic Stability of Heavy-Tailed SGD with General Loss Functions (2 citations)
• Benign Overfitting in Two-layer ReLU Convolutional Neural Networks (2 citations)

and 26 more in the interactive map.

Optimizers · 29 papers

• Learning GFlowNets From Partial Episodes For Improved Convergence And Stability (5 citations)
• Towards Sustainable Learning: Coresets for Data-efficient Deep Learning (4 citations)
• Generalized Polyak Step Size for First Order Optimization with Momentum (3 citations)
• Can Forward Gradient Match Backpropagation? (2 citations)
• DoG is SGD's Best Friend: A Parameter-Free Dynamic Step Size Schedule (2 citations)
• Existence and Estimation of Critical Batch Size for Training Generative Adversarial Networks with Two Time-Scale Update Rule (2 citations)
• Neural Wasserstein Gradient Flows for Discrepancies with Riesz Kernels (2 citations)
• Toward Efficient Gradient-Based Value Estimation (2 citations)
• Achieving High Accuracy with PINNs via Energy Natural Gradient Descent (1 citation)
• Efficient preconditioned stochastic gradient descent for estimation in latent variable models (1 citation)
• Enabling First-Order Gradient-Based Learning for Equilibrium Computation in Markets (1 citation)
• From Perception to Programs: Regularize, Overparameterize, and Amortize (1 citation)

and 17 more in the interactive map.

Distributed and compressed · 10 papers

• Quantized Distributed Training of Large Models with Convergence Guarantees (5 citations)
• Robust Collaborative Learning with Linear Gradient Overhead (1 citation)
• Compressed Decentralized Proximal Stochastic Gradient Method for Nonconvex Composite Problems with Heterogeneous Data
• DADAO: Decoupled Accelerated Decentralized Asynchronous Optimization
• DSGD-CECA: Decentralized SGD with Communication-Optimal Exact Consensus Algorithm
• EF21-P and Friends: Improved Theoretical Communication Complexity for Distributed Optimization with Bidirectional Compression
• Is Consensus Acceleration Possible in Decentralized Optimization over Slowly Time-Varying Networks?
• Optimal Shrinkage for Distributed Second-Order Optimization
• Distributed Stochastic Gradient Descent: Nonconvexity, Nonsmoothness, and Convergence to Local Minima
• Momentum Ensures Convergence of SIGNSGD under Weaker Assumptions