ICML 2020 accepted papers on Optimization
118 of the 1,087 papers accepted at ICML 2020 (10.9%) are on Optimization, compared with 13.4% at ICML 2019. They fall into 4 topics.
See ICML 2020 in the interactive map
Submitting to ICML? Checklist →
Working on something in this area? Describe your idea in scime atlas to see the closest papers by meaning and how crowded the spot has become.
Topics and papers
Nonconvex and smooth optimization · 49 papers
- On Gradient Descent Ascent for Nonconvex-Concave Minimax Problems (183 citations)
- On the Global Convergence Rates of Softmax Policy Gradient Methods (75 citations)
- What is Local Optimality in Nonconvex-Nonconcave Minimax Optimization? (67 citations)
- Implicit Learning Dynamics in Stackelberg Games: Equilibria Characterization, Convergence Analysis, and Empirical Study (38 citations)
- Coresets for Data-efficient Training of Machine Learning Models (37 citations)
- Finite-Time Convergence in Continuous-Time Optimization (21 citations)
- Anderson Acceleration of Proximal Gradient Methods (15 citations)
- Stochastic Hamiltonian Gradient Methods for Smooth Games (14 citations)
- Fine-Grained Analysis of Stability and Generalization for Stochastic Gradient Descent (13 citations)
- The Complexity of Finding Stationary Points with Stochastic Gradient Descent (12 citations)
- On Differentially Private Stochastic Convex Optimization with Heavy-tailed Data (10 citations)
- On the Convergence of Nesterov’s Accelerated Gradient Method in Stochastic Settings (10 citations)
and 37 more in the interactive map.
Optimizers · 29 papers
- SIGUA: Forgetting May Make Learning with Noisy Labels More Robust (59 citations)
- Multi-Task Learning with User Preferences: Gradient Descent with Controlled Ascent in Pareto Optimization (37 citations)
- A Generic First-Order Algorithmic Framework for Bi-Level Programming Beyond Lower-Level Singleton (34 citations)
- Implicit differentiation of Lasso-type models for hyperparameter optimization (25 citations)
- Training Neural Networks for and by Interpolation (22 citations)
- On the Iteration Complexity of Hypergradient Computation (21 citations)
- Non-convex Learning via Replica Exchange Stochastic Gradient MCMC (19 citations)
- AdaScale SGD: A User-Friendly Algorithm for Distributed Training (18 citations)
- Fast and Three-rious: Speeding Up Weak Supervision with Triplet Methods (17 citations)
- Implicit competitive regularization in GANs (17 citations)
- Adaptive Gradient Descent without Descent (16 citations)
- Stochastic Optimization for Regularized Wasserstein Estimators (9 citations)
and 17 more in the interactive map.
Implicit bias and generalization · 25 papers
- Do We Need Zero Training Loss After Achieving Zero Training Error? (59 citations)
- Dynamics of Deep Neural Networks and Neural Tangent Hierarchy (37 citations)
- On the Noisy Gradient Descent that Generalizes as SGD (37 citations)
- The Implicit Regularization of Stochastic Gradient Flow for Least Squares (34 citations)
- The Impact of Neural Network Overparameterization on Gradient Confusion and Stochastic Gradient Descent (31 citations)
- Frequency Bias in Neural Networks for Input of Non-Uniform Density (28 citations)
- The Implicit and Explicit Regularization Effects of Dropout (27 citations)
- Compressive sensing with un-trained neural networks: Gradient descent finds a smooth approximation (24 citations)
- Fractional Underdamped Langevin Dynamics: Retargeting SGD with Momentum under Heavy-Tailed Gradient Noise (22 citations)
- On the Generalization Benefit of Noise in Stochastic Gradient Descent (22 citations)
- Landscape Connectivity and Dropout Stability of SGD Solutions for Over-parameterized Neural Networks (18 citations)
- A Mean Field Analysis Of Deep ResNet And Beyond: Towards Provably Optimization Via Overparameterization From Depth (14 citations)
and 13 more in the interactive map.
Distributed and compressed · 15 papers
- A Unified Theory of Decentralized SGD with Changing Topology and Local Updates (155 citations)
- Is Local SGD Better than Minibatch SGD? (98 citations)
- Acceleration for Compressed Gradient Descent in Distributed and Federated Optimization (64 citations)
- Improving the Sample and Communication Complexity for Decentralized Non-Convex Optimization: Joint Gradient Estimation and Tracking (34 citations)
- Quantized Decentralized Stochastic Learning over Directed Graphs (34 citations)
- Moniqua: Modulo Quantized Communication in Decentralized SGD (30 citations)
- Statistically Preconditioned Accelerated Gradient Method for Distributed Optimization (16 citations)
- Communication-Efficient Distributed Stochastic AUC Maximization with Deep Neural Networks (15 citations)
- Distributed Online Optimization over a Heterogeneous Network with Any-Batch Mirror Descent (11 citations)
- Decentralised Learning with Random Features and Distributed Gradient Descent (8 citations)
- Communication-Efficient Distributed PCA by Riemannian Optimization (6 citations)
- On Efficient Constructions of Checkpoints (4 citations)
and 3 more in the interactive map.
