NeurIPS 2019 accepted papers on Optimization
159 of the 1,429 papers accepted at NeurIPS 2019 (11.1%) are on Optimization, compared with 11.2% at NeurIPS 2018. They fall into 4 topics.
See NeurIPS 2019 in the interactive map
Submitting to NeurIPS? Checklist →
Working on something in this area? Describe your idea in scime atlas to see the closest papers by meaning and how crowded the spot has become.
Topics and papers
Nonconvex and smooth optimization · 67 papers
- Solving a Class of Non-Convex Min-Max Games Using Iterative First Order Methods (138 citations)
- Private Stochastic Convex Optimization with Optimal Rates (77 citations)
- Efficient Algorithms for Smooth Minimax Optimization (75 citations)
- Policy Optimization Provably Converges to Nash Equilibria in Zero-Sum Linear Quadratic Games (72 citations)
- The Step Decay Schedule: A Near Optimal, Geometrically Decaying Learning Rate Procedure For Least Squares (66 citations)
- Understanding the Role of Momentum in Stochastic Gradient Methods (62 citations)
- Finite-Sample Analysis for SARSA with Linear Function Approximation (61 citations)
- SpiderBoost and Momentum: Faster Variance Reduction Algorithms (59 citations)
- Reducing Noise in GAN Training with Variance Reduced Extragradient (56 citations)
- Acceleration via Symplectic Discretization of High-Resolution Differential Equations (53 citations)
- A Linearly Convergent Proximal Gradient Algorithm for Decentralized Optimization (49 citations)
- Competitive Gradient Descent (45 citations)
and 55 more in the interactive map.
Optimizers · 39 papers
- Deep Leakage from Gradients (1,528 citations)
- Lookahead Optimizer: k steps forward, 1 step back (372 citations)
- Differentiable Convex Optimization Layers (124 citations)
- Full-Gradient Representation for Neural Network Visualization (72 citations)
- Efficient Meta Learning via Minibatch Proximal Update (48 citations)
- Stein Variational Gradient Descent With Matrix-Valued Kernels (41 citations)
- Limitations of the empirical Fisher approximation for natural gradient descent (38 citations)
- Information-Theoretic Generalization Bounds for SGLD via Data-Dependent Estimates (36 citations)
- ZO-AdaMM: Zeroth-Order Adaptive Momentum Method for Black-Box Optimization (33 citations)
- Trivializations for Gradient-Based Optimization on Manifolds (31 citations)
- Adaptive Gradient-Based Meta-Learning Methods (22 citations)
- MetaInit: Initializing learning by learning to initialize (22 citations)
and 27 more in the interactive map.
Implicit bias and generalization · 39 papers
- Wide Neural Networks of Any Depth Evolve as Linear Models Under Gradient Descent (290 citations)
- On Lazy Training in Differentiable Programming (249 citations)
- Generalization Bounds of Stochastic Gradient Descent for Wide and Deep Neural Networks (158 citations)
- Towards Explaining the Regularization Effect of Initial Large Learning Rate in Training Neural Networks (121 citations)
- The Convergence Rate of Neural Networks for Learned Functions of Different Frequencies (116 citations)
- On the Convergence Rate of Training Recurrent Neural Networks (111 citations)
- Uniform convergence may be unable to explain generalization in deep learning (111 citations)
- Control Batch Size and Learning Rate to Generalize Well: Theoretical and Empirical Evidence (108 citations)
- SGD on Neural Networks Learns Functions of Increasing Complexity (96 citations)
- An Improved Analysis of Training Over-parameterized Deep Neural Networks (92 citations)
- Regularization Matters: Generalization and Optimization of Neural Nets v.s. their Induced Kernel (70 citations)
- Implicit Regularization in Deep Matrix Factorization (66 citations)
and 27 more in the interactive map.
Distributed and compressed · 14 papers
- Qsparse-local-SGD: Distributed SGD with Quantization, Sparsification and Local Computations (228 citations)
- PowerSGD: Practical Low-Rank Gradient Compression for Distributed Optimization (145 citations)
- Local SGD with Periodic Averaging: Tighter Analysis and Adaptive Synchronization (103 citations)
- Communication-Efficient Distributed Blockwise Momentum SGD with Error-Feedback (67 citations)
- Communication-Efficient Distributed Learning via Lazily Aggregated Quantized Gradients (60 citations)
- DINGO: Distributed Newton-Type Method for Gradient-Norm Optimization (33 citations)
- Double Quantization for Communication-Efficient Distributed Optimization (32 citations)
- An Accelerated Decentralized Stochastic Proximal Algorithm for Finite Sums (25 citations)
- Communication trade-offs for Local-SGD with large step size (19 citations)
- Communication-efficient Distributed SGD with Sketching (16 citations)
- Robust and Communication-Efficient Collaborative Learning (9 citations)
- Optimal Statistical Rates for Decentralised Non-Parametric Regression with Linear Speed-Up (7 citations)
and 2 more in the interactive map.
