Research map / Optimization / Implicit bias and generalization
Parallelizing Stochastic Gradient Descent for Least Squares Regression: Mini-batching, Averaging, and Model Misspecification
Prateek Jain, Sham M. Kakade, Rahul Kidambi, Praneeth Netrapalli, Aaron Sidford
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| increase mini-batch size up to threshold 11 (ablation) | Minibatch tail-averaged SGD · Synthetic Gaussian (d=50, kappa=50) | bias decay rate: batch sizes 1, 3, 11 | Improved |
| increase mini-batch size beyond threshold to 22 (ablation) | Minibatch tail-averaged SGD · Synthetic Gaussian (d=50, kappa=50) | generalization error: 11 -> 22 | Hurt |
| use batch size 11 instead of 1 (ablation) | Tail-averaged SGD · Synthetic Gaussian (d=50, kappa=50) | serial iterations to equal error: 1 -> 11 | Improved |
| add tail-averaging after burn-in versus no averaging (ablation) | Mini-batch SGD with batch 11 · Synthetic Gaussian (d=50, kappa=50) | generalization error | Improved |
| start averaging after burn-in instead of from the start (ablation) | Mini-batch SGD with batch 11 · Synthetic Gaussian (d=50, kappa=50) | generalization error | Improved |
Closest papers by meaning
- On the Utility of Equal Batch Sizes for Inference in Stochastic Gradient Descent (JMLR 2025 · similarity 0.92)
- The Benefits of Implicit Regularization from SGD in Least Squares Problems (NeurIPS 2021 · similarity 0.90)
- Distributed Stochastic Optimization via Adaptive SGD (NeurIPS 2018 · similarity 0.90)
- Gaussian Process Parameter Estimation Using Mini-batch Stochastic Gradient Descent: Convergence Guarantees and Empirical Benefits (JMLR 2022 · similarity 0.90)
- Beating SGD Saturation with Tail-Averaging and Minibatching (NeurIPS 2019 · similarity 0.90)
- Towards Stability and Generalization Bounds in Decentralized Minibatch Stochastic Gradient Descent (AAAI 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
