ICLR 2019 accepted papers on Optimization
43 of the 502 papers accepted at ICLR 2019 (8.6%) are on Optimization, compared with 7.4% at ICLR 2018. They fall into 4 topics.
See ICLR 2019 in the interactive map
Submitting to ICLR? Checklist →
Working on something in this area? Describe your idea in scime atlas to see the closest papers by meaning and how crowded the spot has become.
Topics and papers
Implicit bias and generalization · 16 papers
- Gradient Descent Provably Optimizes Over-parameterized Neural Networks (365 citations)
- A Convergence Analysis of Gradient Descent for Deep Linear Neural Networks (104 citations)
- Gradient descent aligns the layers of deep linear networks (103 citations)
- A Mean Field Theory of Batch Normalization (83 citations)
- Theoretical Analysis of Auto Rate-Tuning by Batch Normalization (67 citations)
- Understanding Straight-Through Estimator in Training Activation Quantized Neural Nets (60 citations)
- On the Relation Between the Sharpest Directions of DNN Loss and the SGD Step Length (50 citations)
- Three Mechanisms of Weight Decay Regularization (49 citations)
- Fluctuation-dissipation relations for stochastic gradient descent (43 citations)
- Deterministic PAC-Bayesian generalization bounds for deep networks via generalizing noise-resilience (37 citations)
- SGD Converges to Global Minimum in Deep Learning via Star-convex Path (36 citations)
- Subgradient Descent Learns Orthogonal Dictionaries (35 citations)
and 4 more in the interactive map.
Optimizers · 15 papers
- Decoupled Weight Decay Regularization (9,419 citations)
- Adaptive Gradient Methods with Dynamic Bound of Learning Rate (182 citations)
- A Variational Inequality Perspective on Generative Adversarial Networks (157 citations)
- Quasi-hyperbolic momentum and Adam for deep learning (87 citations)
- Stochastic Optimization of Sorting Networks via Continuous Relaxations (73 citations)
- AutoLoss: Learning Discrete Schedule for Alternate Optimization (29 citations)
- AdaShift: Decorrelation and Convergence of Adaptive Learning Rate Methods (28 citations)
- Self-Tuning Networks: Bilevel Optimization of Hyperparameters using Structured Best-Response Functions (26 citations)
- Deep Frank-Wolfe For Neural Network Optimization (23 citations)
- Relaxed Quantization for Discretized Neural Networks (16 citations)
- h-detach: Modifying the LSTM Gradient Towards Better Optimization (12 citations)
- Universal Stagewise Learning for Non-Convex Problems with Convergence on Averaged Solutions (8 citations)
and 3 more in the interactive map.
Nonconvex and smooth optimization · 8 papers
- Riemannian Adaptive Optimization Methods (145 citations)
- Optimistic mirror descent in saddle-point problems: Going the extra (gradient) mile (114 citations)
- On the Convergence of A Class of Adam-Type Algorithms for Non-Convex Optimization (93 citations)
- ProxQuant: Quantized Neural Networks via Proximal Operators (71 citations)
- ACCELERATING NONCONVEX LEARNING VIA REPLICA EXCHANGE LANGEVIN DIFFUSION (15 citations)
- A2BCD: Asynchronous Acceleration with Optimal Complexity (10 citations)
- Efficiently testing local optimality and escaping saddles for ReLU networks (4 citations)
- signSGD via Zeroth-Order Oracle
Distributed and compressed · 4 papers
- Local SGD Converges Fast and Communicates Little (208 citations)
- signSGD with Majority Vote is Communication Efficient and Fault Tolerant (124 citations)
- ANYTIME MINIBATCH: EXPLOITING STRAGGLERS IN ONLINE DISTRIBUTED OPTIMIZATION (24 citations)
- Toward Understanding the Impact of Staleness in Distributed Machine Learning (20 citations)
