Research map / Optimization / Optimizers
Stochastic Reweighted Gradient Descent
Ayoub El Hanchi, David Stephens, Chris Maddison
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use SRG's adaptive importance sampling and gradient-norm table instead of uniform SGD sampling | SGD · ijcnn1, w8a, mushrooms, phishing | relative error: about an order of magnitude more accurate | Improved |
| Use SRG instead of SGD | SGD · Synthetic linear regression | relative error: approximately two orders of magnitude more accurate | Improved |
| Use SRG instead of SGD | SGD · ijcnn1, w8a, mushrooms, phishing; decreasing step sizes | relative error | Improved |
| Use SRG instead of SGD with random shuffling | SGD with random shuffling · ijcnn1, w8a, mushrooms, phishing; decreasing step sizes | relative error | Hurt |
| Use SRG instead of SVRG | SVRG · ijcnn1, w8a, mushrooms, phishing; constant step size | relative error | Mixed |
| Use SRG rather than uniform sampling | SGD · Synthetic linear regression | gradient estimator variance: r ≈ 50 | Improved |
| Use SRG's adaptive sampling | SGD · Strongly-convex finite-sum optimization | asymptotic error dependence: σ^2 -> σ_*^2 | Improved |
Closest papers by meaning
- Safe Adaptive Importance Sampling (NeurIPS 2017 · similarity 0.89)
- Stochastic Approximate Gradient Descent via the Langevin Algorithm (AAAI 2020 · similarity 0.89)
- A Variational Analysis of Stochastic Gradient Algorithms (ICML 2016 · similarity 0.89)
- Cheap Bootstrap for Fast Uncertainty Quantification of Stochastic Gradient Descent (JMLR 2026 · similarity 0.88)
- Uncertainty Sampling is Preconditioned Stochastic Gradient Descent on Zero-One Loss (NeurIPS 2018 · similarity 0.88)
- On the Convergence of mSGD and AdaGrad for Stochastic Optimization (ICLR 2022 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
