NeurIPS 2022 accepted papers on Optimization
199 of the 2,901 papers accepted at NeurIPS 2022 (6.9%) are on Optimization, compared with 7.0% at NeurIPS 2021. They fall into 4 topics.
See NeurIPS 2022 in the interactive map
Submitting to NeurIPS? Checklist →
Working on something in this area? Describe your idea in scime atlas to see the closest papers by meaning and how crowded the spot has become.
Topics and papers
Nonconvex and smooth optimization · 82 papers
- On Convergence of FedProx: Local Dissimilarity Invariant Bounds, Non-smoothness and Beyond (30 citations)
- A framework for bilevel optimization that enables stochastic and global variance reduction algorithms (11 citations)
- Optimal Algorithms for Decentralized Stochastic Variational Inequalities (9 citations)
- Accelerated Primal-Dual Gradient Method for Smooth and Convex-Concave Saddle-Point Problems with Bilinear Coupling (8 citations)
- Clipped Stochastic Methods for Variational Inequalities with Heavy-Tailed Noise (6 citations)
- Distributed Distributionally Robust Optimization with Non-Convex Objectives (6 citations)
- Global Convergence and Stability of Stochastic Gradient Descent (6 citations)
- A Unified Convergence Theorem for Stochastic Optimization Methods (5 citations)
- Large-scale Optimization of Partial AUC in a Range of False Positive Rates (5 citations)
- On the convergence of policy gradient methods to Nash equilibria in general stochastic games (5 citations)
- Quasi-Newton Methods for Saddle Point Problems (5 citations)
- Reproducibility in Optimization: Theoretical Framework and Limits (5 citations)
and 70 more in the interactive map.
Implicit bias and generalization · 51 papers
- High-dimensional limit theorems for SGD: Effective dynamics and critical scaling (18 citations)
- Make Sharpness-Aware Minimization Stronger: A Sparsified Perturbation Approach (17 citations)
- High-dimensional Asymptotics of Feature Learning: How One Gradient Step Improves the Representation (10 citations)
- Phase diagram of Stochastic Gradient Descent in high-dimensional two-layer neural networks (10 citations)
- Stability Analysis and Generalization Bounds of Adversarial Training (9 citations)
- Beyond accuracy: generalization properties of bio-plausible temporal credit assignment rules (7 citations)
- When Do Flat Minima Optimizers Work? (6 citations)
- Random Sharpness-Aware Minimization (5 citations)
- Analyzing Sharpness along GD Trajectory: Progressive Sharpening and Edge of Stability (4 citations)
- Data-Efficient Augmentation for Training Neural Networks (4 citations)
- Do Residual Neural Networks discretize Neural Ordinary Differential Equations? (4 citations)
- On the Double Descent of Random Features Models Trained with SGD (4 citations)
and 39 more in the interactive map.
Optimizers · 49 papers
- Deep Differentiable Logic Gate Networks (18 citations)
- Tight Mutual Information Estimation With Contrastive Fenchel-Legendre Optimization (10 citations)
- Privacy of Noisy Stochastic Gradient Descent: More Iterations without More Privacy Loss (9 citations)
- Nonlinear Sufficient Dimension Reduction with a Stochastic Neural Network (8 citations)
- VectorAdam for Rotation Equivariant Geometry Optimization (8 citations)
- A Theoretical Framework for Inference Learning (7 citations)
- End-to-end Stochastic Optimization with Energy-based Model (7 citations)
- Adam Can Converge Without Any Modification On Update Rules (6 citations)
- BOME! Bilevel Optimization Made Easy: A Simple First-Order Approach (6 citations)
- GAL: Gradient Assisted Learning for Decentralized Multi-Organization Collaborations (6 citations)
- Exploring the Algorithm-Dependent Generalization of AUPRC Optimization with List Stability (5 citations)
- Gradient Descent: The Ultimate Optimizer (5 citations)
and 37 more in the interactive map.
Distributed and compressed · 17 papers
- Sharper Convergence Guarantees for Asynchronous SGD for Distributed and Federated Learning (18 citations)
- Decentralized Local Stochastic Extra-Gradient for Variational Inequalities (13 citations)
- A Communication-Efficient Distributed Gradient Clipping Algorithm for Training Deep Neural Networks (6 citations)
- Asynchronous SGD Beats Minibatch SGD Under Arbitrary Delays (6 citations)
- BEER: Fast $O(1/T)$ Rate for Decentralized Nonconvex Optimization with Communication Compression (6 citations)
- Rate-Distortion Theoretic Bounds on Generalization Error for Distributed Learning (4 citations)
- Distributed Methods with Compressed Communication for Solving Variational Inequalities, with Theoretical Guarantees (3 citations)
- EF-BV: A Unified Theory of Error Feedback and Variance Reduction Mechanisms for Biased and Unbiased Compression in Distributed Optimization (3 citations)
- Fine-tuning Language Models over Slow Networks using Activation Quantization with Guarantees (2 citations)
- Theoretically Better and Numerically Faster Distributed Optimization with Smoothness-Aware Quantization Techniques (1 citation)
- Acceleration in Distributed Sparse Regression
- Escaping Saddle Points with Bias-Variance Reduced Local Perturbed SGD for Communication Efficient Nonconvex Distributed Learning
and 5 more in the interactive map.
