Research map / Optimization / Nonconvex and smooth optimization
Stochastic Optimization with Laggard Data Pipelines
Naman Agarwal, Rohan Anil, Tomer Koren, Kunal Talwar, Cyril Zhang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Take K gradient steps on each minibatch instead of one | SGD · Stochastic convex optimization | curvature term in convergence rate: \frac{\beta D^2}{T} -> \frac{\beta D^2}{KT} | Improved |
| Take K gradient steps on each minibatch instead of one | SGD · Stochastic convex optimization | statistical term in convergence rate: \frac{\rho D}{\sqrt{BT}} -> \frac{\rho D}{\sqrt{BT}} | No clear effect |
| Echo gradient descent with a proximal regularizer | Proximal GD · Stochastic convex optimization | dependence of the chosen step size on K | Improved |
| Use K accelerated gradient steps on each batch | Stochastic AGD · Quadratic losses | curvature term in convergence rate: \frac{\beta D^2}{T^2} -> \frac{\beta D^2}{K^2T^2} | Improved |
| Increase the echoing factor K | Data-echoed SGD · CoverType and MNIST logistic regression | fresh-sample convergence time in the variance-dominated regime | No clear effect |
| Increase the echoing factor K | Data-echoed SGD · CoverType and MNIST logistic regression | convergence time in the curvature-dominated regime | Improved |
| Tune the learning rate inversely with K | Data-echoed SGD · CoverType and MNIST logistic regression, variance-dominated regime | convergence time | Improved |
Closest papers by meaning
- Faster Stochastic Optimization with Arbitrary Delays via Adaptive Asynchronous Mini-Batching (ICML 2025 · similarity 0.89)
- Never Go Full Batch (in Stochastic Convex Optimization) (NeurIPS 2021 · similarity 0.88)
- SGD: General Analysis and Improved Rates (ICML 2019 · similarity 0.88)
- Perfect Parallelization in Mini-Batch SGD with Classical Momentum Acceleration (NeurIPS 2026 · similarity 0.87)
- Stochastic Optimization with Arbitrary Recurrent Data Sampling (ICML 2024 · similarity 0.86)
- Stochastic Zeroth-Order Optimization Under Heavy-Tailed Noise (NeurIPS 2026 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
