Research map / Optimization / Nonconvex and smooth optimization
On the Theory of Variance Reduction for Stochastic Gradient Monte Carlo
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use variance reduction rather than plain stochastic gradients | SG-MCMC in the high-accuracy regime · W2 convergence, tall-data regime | computational complexity: SGLD: \kappa^2 d/\epsilon^2; SAGA-LD: N\!+\!\kappa^{\frac{3}{2}}\sqrt{d}/\epsilon | Helped |
| Use SGLD instead of variance reduction when low accuracy and less than one data pass suffice | SG-MCMC · W2 convergence, low-accuracy regime | computational complexity | Helped |
| Use SAGA-LD instead of SGLD | SG-MCMC · Bayesian logistic regression, UCI datasets | passes through data to target log held-out probability | Mixed |
| Use SAGA-LD or SVRG-LD instead of SGLD | SG-MCMC · Synthetic log-normal data | mean squared error | Helped |
| Use CV-LD | SG-MCMC · Synthetic log-normal data | mean squared error | Mixed |
| Use SVRG-LD Option I rather than SAGA-LD (ablation) | Langevin sampling · W2 convergence, tall-data regime | mixing time: \kappa^3d/(n\epsilon^2) vs. \kappa^{\frac{3}{2}}\sqrt{d}/(n\epsilon) | Hurt |
| Use SVRG-LD Option II rather than SAGA-LD (ablation) | Langevin sampling · W2 convergence, tall-data regime | computational complexity: N\!+\!\kappa^{\frac{5}{3}}N^{\frac{1}{6}}\sqrt{d}/\epsilon vs. N\!+\!\kappa^{\frac{3}{2}}\sqrt{d}/\epsilon | Hurt |
| Use SVRG-LD Option II rather than Option I (ablation) | SVRG-LD · W2 convergence, tall-data regime | mixing time: \kappa^3d/(n\epsilon^2) vs. \kappa^{\frac{11}{6}}\sqrt{d}/(N^{\frac{2}{3}}\epsilon) | Helped |
| Use CV-ULD rather than CV-LD | Control-variate Langevin sampling · W2 convergence, tall-data regime | mixing time: \kappa^3d/(N\epsilon^2) vs. \kappa^{\frac{5}{2}}\sqrt{d}/(\sqrt{N}\epsilon) | Helped |
| Use CV-ULD rather than SAGA-LD | Variance-reduced Langevin sampling · Bayesian logistic regression, UCI datasets | passes through data to target log held-out probability | Mixed |
| Rely on stochastic mini-batches without stored history | SGLD · Synthetic log-normal data with mini-batch size 10 | mean squared error | Hurt |
| Update the SVRG control point only periodically rather than using SAGA's per-sample gradient history | Variance-reduced Langevin sampling · Bayesian logistic regression, UCI datasets | passes through data to target log held-out probability | Hurt |
Closest papers by meaning
- Variance Reduction in Stochastic Gradient Langevin Dynamics (NeurIPS 2016 · similarity 0.92)
- Stochastic Variance-Reduced Hamilton Monte Carlo Methods (ICML 2018 · similarity 0.91)
- Stochastic Gradient Hamiltonian Monte Carlo Methods with Recursive Variance Reduction (NeurIPS 2019 · similarity 0.91)
- Variance Reduction in Stochastic Particle-Optimization Sampling (ICML 2020 · similarity 0.91)
- Stochastic variance-reduced Gaussian variational inference on the Bures-Wasserstein manifold (ICLR 2025 · similarity 0.88)
- Non-asymptotic Error Bounds in $\mathcal{W}_2$-Distance with Sqrt(d) Dimension Dependence and First Order Convergence for Langevin Monte Carlo beyond Log-Concavity (ICML 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.