Research map / Optimization / Implicit bias and generalization
On the Utility of Equal Batch Sizes for Inference in Stochastic Gradient Descent
Rahul Singh, Abhinek Shukla, Dootika Vats
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use equal batch sizes (EBS) instead of increasing batch sizes (IBS) (ablation) | batch-means covariance estimator for ASGD · Linear regression, d=5 and d=20, 1000 replications | relative Frobenius error, coverage, region volume | Improved |
| add lugsail bias correction to EBS (Sigma_L = 2 Sigma_2b - Sigma_b) (ablation) | EBS batch-means estimator · Linear regression and LAD regression | bias, coverage, volume | Improved |
| use polynomial batch size floor(c n^beta) instead of power-of-two doubling batch size (ablation) | EBS estimator · Linear regression d=5 | monotonic behavior in sample size | Improved |
| use simultaneous hyper-rectangular region C(z*) rather than ellipsoid with IBS | IBS estimator · Linear regression and LAD regression | coverage versus volume | Mixed |
| use batch-size exponent beta=(1+alpha)/2 instead of numerically optimal value for finite n (ablation) | EBS estimator · alpha=0.51, n in thousands | mean-square error bound: 0.66 | Mixed |
| classify using lower confidence bound of predicted probability with EBS standard errors (ablation) | logistic classifier trained by ASGD · Santander, Covertype, Spambase, diabetes datasets | test misclassification rate | Improved |
Closest papers by meaning
- Parallelizing Stochastic Gradient Descent for Least Squares Regression: Mini-batching, Averaging, and Model Misspecification (JMLR 2018 · similarity 0.92)
- Statistical Inference Using SGD (AAAI 2018 · similarity 0.91)
- Disentangling the Mechanisms Behind Implicit Regularization in SGD (ICLR 2023 · similarity 0.91)
- On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minima (ICLR 2017 · similarity 0.90)
- Subsampling is not Magic: Why Large Batch Sizes Work for Differentially Private Stochastic Optimisation (ICML 2024 · similarity 0.89)
- Statistical Guarantees for High-Dimensional Stochastic Gradient Descent (NeurIPS 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
