ICML 2017 accepted papers on Optimization
60 of the 435 papers accepted at ICML 2017 (13.8%) are on Optimization, compared with 12.7% at ICML 2016. They fall into 4 topics.
See ICML 2017 in the interactive map
Submitting to ICML? Checklist →
Working on something in this area? Describe your idea in scime atlas to see the closest papers by meaning and how crowded the spot has become.
Topics and papers
Nonconvex and smooth optimization · 28 papers
- How to Escape Saddle Points Efficiently (219 citations)
- Prox-PDA: The Proximal Primal-Dual Algorithm for Fast Distributed Nonconvex Optimization and Learning Over Networks (114 citations)
- Variants of RMSProp and Adagrad with Logarithmic Regret Bounds (109 citations)
- SARAH: A Novel Method for Machine Learning Problems Using Stochastic Recursive Gradient (81 citations)
- Stochastic Variance Reduction Methods for Policy Evaluation (62 citations)
- Convergence Analysis of Proximal Gradient with Momentum for Nonconvex Optimization (56 citations)
- Global optimization of Lipschitz functions (52 citations)
- Sub-sampled Cubic Regularization for Non-convex Optimization (43 citations)
- Efficient Regret Minimization in Non-Convex Games (37 citations)
- Natasha: Faster Non-Convex Stochastic Optimization via Strongly Non-Convex Parameter (34 citations)
- “Convex Until Proven Guilty”: Dimension-Free Acceleration of Gradient Descent on Non-Convex Functions (33 citations)
- Stochastic Convex Optimization: Faster Local Growth Implies Faster Global Convergence (31 citations)
and 16 more in the interactive map.
Optimizers · 22 papers
- Neural Optimizer Search with Reinforcement Learning (259 citations)
- Learning to Learn without Gradient Descent by Gradient Descent (174 citations)
- On orthogonality and learning recurrent networks with long term dependencies (154 citations)
- Failures of Gradient-Based Deep Learning (66 citations)
- Forward and Reverse Gradient-Based Hyperparameter Optimization (53 citations)
- Learning Gradient Descent: Better Generalization and Longer Horizons (49 citations)
- Efficient Orthogonal Parametrisation of Recurrent Neural Networks Using Householder Reflections (47 citations)
- Deep Latent Dirichlet Allocation with Topic-Layer-Adaptive Stochastic Gradient Riemannian MCMC (43 citations)
- Understanding Synthetic Gradients and Decoupled Neural Interfaces (39 citations)
- Developing Bug-Free Machine Learning Systems With Formal Mathematics (38 citations)
- End-to-End Learning for Structured Prediction Energy Networks (27 citations)
- Stochastic Bouncy Particle Sampler (24 citations)
and 10 more in the interactive map.
Distributed and compressed · 5 papers
- Gradient Coding: Avoiding Stragglers in Distributed Learning (334 citations)
- Optimal Algorithms for Smooth and Strongly Convex Distributed Optimization in Networks (192 citations)
- Asynchronous Stochastic Gradient Descent with Delay Compensation (158 citations)
- Efficient Distributed Learning with Sparsity (45 citations)
- Adaptive Consensus ADMM for Distributed Optimization (37 citations)
Implicit bias and generalization · 5 papers
- The Loss Surface of Deep and Wide Neural Networks (168 citations)
- Sharp Minima Can Generalize For Deep Nets (134 citations)
- Globally Optimal Gradient Descent for a ConvNet with Gaussian Inputs (71 citations)
- An Analytical Formula of Population Gradient for two-layered ReLU network and its Applications in Convergence and Critical Point Analysis (69 citations)
- Provable Alternating Gradient Descent for Non-negative Matrix Factorization with Strong Correlations (14 citations)
