Research map / Optimization / Implicit bias and generalization
Cheap Bootstrap for Fast Uncertainty Quantification of Stochastic Gradient Descent
Henry Lam, Zitong Wang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| increase number of resamples B from 3 to 10 (ablation) | COfB ASGD · Linear regression, Toeplitz, n=1e5, d=200 | average interval length (x1e-2): 14.44 -> 8.44 | Improved |
| increase number of resamples B from 3 to 10 (ablation) | COnB · Linear regression, Toeplitz, n=1e5, d=200 | average interval length (x1e-2): 21.04 -> 15.61 | Improved |
| increase B from 3 to 10 (ablation) | COfB/COnB · interval width vs normal | t-interval relative width: 49.6% wider -> 10.9% wider | Improved |
| increase B from 3 to 10 (ablation) | COfB ASGD · Linear regression, Toeplitz, n=1e5, d=200 | coverage probability (%): 94.41 -> 94.01 | No clear effect |
| increase B from 3 to 10 in sparse regression (ablation) | COfB ASGD · Sparse linear regression p=3, in T* | average interval length (x1e-2): 2.75 -> 0.86 | Improved |
| use averaging (ASGD) instead of last-iterate SGD (ablation) | COfB B=3 · Linear regression, Toeplitz, n=1e5, d=5 | average interval length (x1e-2): 6.93 -> 3.06 | Improved |
| use cheap bootstrap B=3 instead of delta method (ablation) | SGD CI · Linear regression, Toeplitz, n=1e5, d=200 | coverage probability (%): 37.16 -> 94.41 | Improved |
| increase dimension from 20 to 200 | delta method · Linear regression, Toeplitz, n=1e5 | coverage probability (%): 93.23 -> 37.16 | Hurt |
| use cheap bootstrap B=3 instead of HiGrad (ablation) | SGD CI · Logistic regression, Toeplitz, n=1e5, d=200 | coverage probability (%): 57.02 -> 93.79 | Improved |
| reduce online bootstrap B from 100 to 10 (ablation) | online bootstrap · Linear regression, Toeplitz, n=1e5, d=200 | coverage probability (%): 96.58 -> 93.33 | No clear effect |
| use COnB B=3 instead of online bootstrap B=100 (ablation) | SGD CI · Linear regression runtime d=10 | runtime (seconds): 45.90 -> 1.75 | Improved |
Closest papers by meaning
- Online Bootstrap Confidence Intervals for the Stochastic Gradient Descent Estimator (JMLR 2018 · similarity 0.92)
- Parallelizing Stochastic Gradient Descent for Least Squares Regression: Mini-batching, Averaging, and Model Misspecification (JMLR 2018 · similarity 0.90)
- A Variational Analysis of Stochastic Gradient Algorithms (ICML 2016 · similarity 0.89)
- Stochastic Reweighted Gradient Descent (ICML 2022 · similarity 0.88)
- Demystifying SGD with Doubly Stochastic Gradients (ICML 2024 · similarity 0.88)
- Nonasymptotic Analysis of Stochastic Gradient Descent with the Richardson–Romberg Extrapolation (ICLR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
