ICML 2019 accepted papers on Optimization
104 of the 774 papers accepted at ICML 2019 (13.4%) are on Optimization, compared with 13.8% at ICML 2018. They fall into 4 topics.
See ICML 2019 in the interactive map
Submitting to ICML? Checklist →
Working on something in this area? Describe your idea in scime atlas to see the closest papers by meaning and how crowded the spot has become.
Topics and papers
Nonconvex and smooth optimization · 43 papers
- AdaGrad Stepsizes: Sharp Convergence Over Nonconvex Landscapes (170 citations)
- Plug-and-Play Methods Provably Converge with Properly Trained Denoisers (99 citations)
- Alternating Minimizations Converge to Second-Order Optimal Solutions (37 citations)
- Global Convergence of Block Coordinate Descent in Deep Learning (32 citations)
- Riemannian adaptive stochastic gradient algorithms on matrix manifolds (28 citations)
- Nonconvex Variance Reduced Optimization with Arbitrary Sampling (25 citations)
- Toward Controlling Discrimination in Online Ad Auctions (25 citations)
- SGD without Replacement: Sharper Rates for General Smooth Convex Functions (23 citations)
- Escaping Saddle Points with Adaptive Gradient Methods (22 citations)
- Estimate Sequences for Variance-Reduced Stochastic Composite Optimization (19 citations)
- Faster Stochastic Alternating Direction Method of Multipliers for Nonconvex Optimization (19 citations)
- Anytime Online-to-Batch, Optimism and Acceleration (18 citations)
and 31 more in the interactive map.
Optimizers · 30 papers
- Sever: A Robust Meta-Algorithm for Stochastic Optimization (142 citations)
- Adaptive Stochastic Natural Gradient Method for One-Shot Neural Architecture Search (49 citations)
- Provable Guarantees for Gradient-Based Meta-Learning (46 citations)
- Adaptive and Safe Bayesian Optimization in High Dimensions via One-Dimensional Subspaces (45 citations)
- Understanding and correcting pathologies in the training of learned optimizers (45 citations)
- Learning a Compressed Sensing Measurement Matrix via Gradient Unrolling (40 citations)
- SGD: General Analysis and Improved Rates (35 citations)
- Beyond Backprop: Online Alternating Minimization with Auxiliary Variables (34 citations)
- Differentiable Linearized ADMM (25 citations)
- Guided evolutionary strategies: augmenting random search with surrogate gradients (25 citations)
- Multi-objective training of Generative Adversarial Networks with multiple discriminators (25 citations)
- Bayesian Optimization Meets Bayesian Optimal Stopping (23 citations)
and 18 more in the interactive map.
Implicit bias and generalization · 21 papers
- A Convergence Theory for Deep Learning via Over-Parameterization (688 citations)
- Fine-Grained Analysis of Optimization and Generalization for Overparameterized Two-Layer Neural Networks (220 citations)
- Gradient Descent Finds Global Minima of Deep Neural Networks (192 citations)
- An Investigation into Neural Net Optimization via Hessian Eigenvalue Density (93 citations)
- A Tail-Index Analysis of Stochastic Gradient Noise in Deep Neural Networks (89 citations)
- Overparameterized Nonlinear Learning: Gradient Descent Takes the Shortest Path? (88 citations)
- The Anisotropic Noise in Stochastic Gradient Descent: Its Behavior of Escaping from Sharp Minima and Regularization Effects (86 citations)
- On Symmetric Losses for Learning from Corrupted Labels (59 citations)
- Width Provably Matters in Optimization for Deep Linear Neural Networks (45 citations)
- Learning-to-Learn Stochastic Gradient Descent with Biased Regularization (40 citations)
- Why do Larger Models Generalize Better? A Theoretical Perspective via the XOR Problem (25 citations)
- Understanding Impacts of High-Order Loss Approximations and Features in Deep Learning Interpretation (24 citations)
and 9 more in the interactive map.
Distributed and compressed · 10 papers
- Decentralized Stochastic Optimization and Gossip Algorithms with Compressed Communication (177 citations)
- Error Feedback Fixes SignSGD and other Gradient Compression Schemes (147 citations)
- On the Linear Speedup Analysis of Communication Efficient Momentum SGD for Distributed Non-Convex Optimization (138 citations)
- Stochastic Gradient Push for Distributed Deep Learning (134 citations)
- DoubleSqueeze: Parallel Stochastic Gradient Descent with Double-pass Error-Compensated Compression (114 citations)
- Zeno: Distributed Stochastic Gradient Descent with Suspicion-based Fault-tolerance (83 citations)
- Trading Redundancy for Communication: Speeding up Distributed SGD for Non-convex Optimization (47 citations)
- On the Complexity of Approximating Wasserstein Barycenters (38 citations)
- On the Computation and Communication Complexity of Parallel SGD with Dynamic Batch Sizes for Stochastic Non-Convex Optimization (27 citations)
- Defending Against Saddle Point Attack in Byzantine-Robust Distributed Learning (8 citations)
