NeurIPS 2018 accepted papers on Optimization
113 of the 1,008 papers accepted at NeurIPS 2018 (11.2%) are on Optimization, compared with 10.3% at NeurIPS 2017. They fall into 4 topics.
See NeurIPS 2018 in the interactive map
Submitting to NeurIPS? Checklist →
Working on something in this area? Describe your idea in scime atlas to see the closest papers by meaning and how crowded the spot has become.
Topics and papers
Nonconvex and smooth optimization · 48 papers
- SPIDER: Near-Optimal Non-Convex Optimization via Stochastic Path-Integrated Differential Estimator (243 citations)
- Stochastic Cubic Regularization for Fast Nonconvex Optimization (99 citations)
- Zeroth-Order Stochastic Variance Reduction for Nonconvex Optimization (96 citations)
- Natasha 2: Faster Non-Convex Optimization Than SGD (89 citations)
- A Simple Proximal Stochastic Gradient Method for Nonsmooth Nonconvex Optimization (66 citations)
- NEON2: Finding Local Minima via First-Order Oracles (64 citations)
- Stochastic Nested Variance Reduction for Nonconvex Optimization (64 citations)
- Direct Runge-Kutta Discretization Achieves Acceleration (55 citations)
- Graph Oracle Models, Lower Bounds, and Gaps for Parallel Stochastic Optimization (51 citations)
- How To Make the Gradients Small Stochastically: Even Faster Convex and Nonconvex SGD (38 citations)
- SEGA: Variance Reduction via Gradient Sketching (37 citations)
- Escaping Saddle Points in Constrained Optimization (28 citations)
and 36 more in the interactive map.
Optimizers · 26 papers
- Adaptive Methods for Nonconvex Optimization (219 citations)
- Fast Approximate Natural Gradient Descent in a Kronecker Factored Eigenbasis (68 citations)
- Training Deep Models Faster with Robust, Approximate Importance Sampling (62 citations)
- On gradient regularizers for MMD GANs (52 citations)
- Training Neural Networks Using Features Replay (52 citations)
- The promises and pitfalls of Stochastic Gradient Langevin Dynamics (44 citations)
- A Stein variational Newton method (39 citations)
- Evolutionary Stochastic Gradient Descent for Optimization of Deep Neural Networks (37 citations)
- Exact natural gradient in deep linear networks and its application to the nonlinear case (36 citations)
- Gradient Descent for Spiking Neural Networks (36 citations)
- L4: Practical loss-based stepsize adaptation for deep learning (33 citations)
- Stochastic Expectation Maximization with Variance Reduction (30 citations)
and 14 more in the interactive map.
Implicit bias and generalization · 21 papers
- Neural Tangent Kernel: Convergence and Generalization in Neural Networks (1,345 citations)
- Visualizing the Loss Landscape of Neural Nets (606 citations)
- Learning Overparameterized Neural Networks via Stochastic Gradient Descent on Structured Data (273 citations)
- Understanding Batch Normalization (208 citations)
- On the Global Convergence of Gradient Descent for Over-parameterized Models using Optimal Transport (173 citations)
- How SGD Selects the Global Minima in Over-parameterized Learning: A Dynamical Stability Perspective (69 citations)
- Parameters as interacting particles: long time convergence and asymptotic error scaling of neural networks (52 citations)
- The Spectrum of the Fisher Information Matrix of a Single-Hidden-Layer Neural Network (41 citations)
- Are ResNets Provably Better than Linear Predictors? (38 citations)
- Implicit Bias of Gradient Descent on Linear Convolutional Networks (35 citations)
- Statistical Optimality of Stochastic Gradient Descent on Hard Learning Problems through Multiple Passes (35 citations)
- Learning with SGD and Random Features (32 citations)
and 9 more in the interactive map.
Distributed and compressed · 18 papers
- Sparsified SGD with Memory (221 citations)
- Gradient Sparsification for Communication-Efficient Distributed Optimization (205 citations)
- ATOMO: Communication-efficient Learning via Atomic Sparsification (198 citations)
- Communication Compression for Decentralized Training (179 citations)
- Byzantine Stochastic Gradient Descent (172 citations)
- A Linear Speedup Analysis of Distributed Deep Learning with Sparse and Quantized Communication (120 citations)
- Optimal Algorithms for Non-Smooth Distributed Optimization in Networks (76 citations)
- Decentralize and Randomize: Faster Algorithm for Wasserstein Barycenters (49 citations)
- GIANT: Globally Improved Approximate Newton Method for Distributed Optimization (30 citations)
- LAG: Lazily Aggregated Gradient for Communication-Efficient Distributed Learning (19 citations)
- BML: A High-performance, Low-cost Gradient Synchronization Algorithm for DML Training (18 citations)
- The Effect of Network Width on the Performance of Large-batch Training (14 citations)
and 6 more in the interactive map.
