Research map / Optimization / Optimizers
Approximation Based Variance Reduction for Reparameterization Gradients
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add a learned quadratic control variate to reparameterization gradients | SGVI with diagonal plus low-rank Gaussian variational distributions · Hierarchical regression, logistic regression, and Bayesian neural network | optimization convergence | Helped |
| Use the learned quadratic control variate instead of the Taylor-based control variate | SGVI with non-factorized Gaussian variational distributions · Hierarchical regression, logistic regression, and Bayesian neural network | optimization convergence | Helped |
| Add a learned quadratic control variate to reparameterization gradients | Gradient estimation with diagonal plus low-rank Gaussians · Three probabilistic models | gradient estimator variance | Helped |
| Use the Taylor-based control variate instead of no control variate | SGVI with non-factorized Gaussian variational distributions · Hierarchical regression, logistic regression, and Bayesian neural network | optimization convergence | No clear effect |
| Use the learned quadratic control variate instead of the Taylor-based control variate | SGVI with diagonal Gaussian variational distributions · Probabilistic models in the experiments | optimization convergence | No clear effect |
| Use the gradient-matching proxy objective rather than directly minimizing estimator variance (ablation) | Quadratic control variate fitting · Bayesian logistic regression variance comparison | gradient estimator variance | No clear effect |
| Use a control-variate rank that is too low (ablation) | Approximation-based CV with diagonal plus low-rank Gaussian variational distributions · Logistic regression and other evaluated models | optimization performance | Hurt |
Closest papers by meaning
- The Generalized Reparameterization Gradient (NeurIPS 2016 · similarity 0.93)
- Reparameterization Gradient for Non-differentiable Models (NeurIPS 2018 · similarity 0.92)
- Least squares variational inference (NeurIPS 2025 · similarity 0.88)
- Stochastic variance-reduced Gaussian variational inference on the Bures-Wasserstein manifold (ICLR 2025 · similarity 0.88)
- Rao-Blackwellised Reparameterisation Gradients (NeurIPS 2025 · similarity 0.87)
- Natural Gradient VI: Guarantees for Non-Conjugate Models (NeurIPS 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.