NeurIPS 2023 accepted papers on Optimization

176 of the 3,584 papers accepted at NeurIPS 2023 (4.9%) are on Optimization, compared with 6.9% at NeurIPS 2022. They fall into 4 topics.

See NeurIPS 2023 in the interactive map

Working on something in this area? Describe your idea in scime atlas to see the closest accepted papers by meaning and how crowded the spot has become.

Topics and papers

Nonconvex and smooth optimization · 69 papers

Convex and Non-convex Optimization Under Generalized Smoothness (4 citations)
Oracle Complexity of Single-Loop Switching Subgradient Methods for Non-Smooth Weakly Convex Functional Constrained Optimization (4 citations)
Accelerated Zeroth-order Method for Non-Smooth Stochastic Convex Optimization Problem with Infinite Variance (3 citations)
Adaptive SGD with Polyak stepsize and Line-search: Robust Convergence and Variance Reduction (3 citations)
An Alternating Optimization Method for Bilevel Problems under the Polyak-Łojasiewicz Condition (3 citations)
Continuous-time Analysis of Anchor Acceleration (3 citations)
Fine-Grained Theoretical Analysis of Federated Zeroth-Order Optimization (3 citations)
Last-Iterate Convergent Policy Gradient Primal-Dual Methods for Constrained MDPs (3 citations)
A Smooth Binary Mechanism for Efficient Private Continual Observation (2 citations)
Contextual Stochastic Bilevel Optimization (2 citations)
Faster Differentially Private Convex Optimization via Second-Order Methods (2 citations)
First Order Methods with Markovian Noise: from Acceleration to Variational Inequalities (2 citations)

and 57 more in the interactive map.

Optimizers · 47 papers

FAMO: Fast Adaptive Multitask Optimization (13 citations)
Meta-AdaM: An Meta-Learned Adaptive Optimizer with Momentum for Few-Shot Learning (5 citations)
A Guide Through the Zoo of Biased SGD (4 citations)
Data-driven Optimal Filtering for Linear Systems with Unknown Noise Covariances (4 citations)
The Behavior and Convergence of Local Bayesian Optimization (4 citations)
Resetting the Optimizer in Deep RL: An Empirical Study (3 citations)
Towards Understanding the Dynamics of Gaussian-Stein Variational Gradient Descent (3 citations)
Direction-oriented Multi-objective Learning: Simple and Provable Stochastic Algorithms (2 citations)
DoWG Unleashed: An Efficient Universal Parameter-Free Gradient Descent Method (2 citations)
Estimating the Rate-Distortion Function by Wasserstein Gradient Descent (2 citations)
Gradient Flossing: Improving Gradient Descent through Dynamic Control of Jacobians (2 citations)
How to Scale Your EMA (2 citations)

and 35 more in the interactive map.

Implicit bias and generalization · 47 papers

Stochastic Collapse: How Gradient Noise Attracts SGD Dynamics Towards Simpler Subnetworks (6 citations)
Transformers learn to implement preconditioned gradient descent for in-context learning (6 citations)
Implicit Bias of Gradient Descent for Two-layer ReLU and Leaky ReLU Networks on Nearly-orthogonal Data (4 citations)
Why Does Sharpness-Aware Minimization Generalize Better Than SGD? (4 citations)
Complex-valued Neurons Can Learn More but Slower than Real-valued Neurons via Gradient Descent (3 citations)
Path Regularization: A Convexity and Sparsity Inducing Regularization for Parallel ReLU Networks (3 citations)
Provable Guarantees for Nonlinear Feature Learning in Three-Layer Neural Networks (3 citations)
(S)GD over Diagonal Linear Networks: Implicit bias, Large Stepsizes and Edge of Stability (2 citations)
Global Convergence Analysis of Local SGD for Two-layer Neural Network without Overparameterization (2 citations)
On skip connections and normalisation layers in deep optimisation (2 citations)
A Unified Fast Gradient Clipping Framework for DP-SGD (1 citation)
Abide by the law and follow the flow: conservation laws for gradient flows (1 citation)