AI Paper Atlas

Research map / Optimization / Implicit bias and generalization

Parallelizing Stochastic Gradient Descent for Least Squares Regression: Mini-batching, Averaging, and Model Misspecification

Prateek Jain, Sham M. Kakade, Rahul Kidambi, Praneeth Netrapalli, Aaron Sidford

JMLR 2018 · Optimization › Implicit bias and generalization · approach: SGD noise and flat minima · 104 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
increase mini-batch size up to threshold 11 (ablation)Minibatch tail-averaged SGD · Synthetic Gaussian (d=50, kappa=50)bias decay rate: batch sizes 1, 3, 11Improved
increase mini-batch size beyond threshold to 22 (ablation)Minibatch tail-averaged SGD · Synthetic Gaussian (d=50, kappa=50)generalization error: 11 -> 22Hurt
use batch size 11 instead of 1 (ablation)Tail-averaged SGD · Synthetic Gaussian (d=50, kappa=50)serial iterations to equal error: 1 -> 11Improved
add tail-averaging after burn-in versus no averaging (ablation)Mini-batch SGD with batch 11 · Synthetic Gaussian (d=50, kappa=50)generalization errorImproved
start averaging after burn-in instead of from the start (ablation)Mini-batch SGD with batch 11 · Synthetic Gaussian (d=50, kappa=50)generalization errorImproved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy