Research map / Probabilistic and scientific modeling / Variational and generative sampling
The Generalized Reparameterization Gradient
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace BBVI gradient with G-REP single-sample reparameterization-plus-correction gradient | gamma/beta mean-field VI · Olivetti, NIPS, MNIST, Omniglot | time per iteration: roughly 10x+ faster than BBVI | Helped |
| Use G-REP instead of ADVI (gamma/beta vs log-normal/logit-normal family) | mean-field VI on sparse models · MNIST | ELBO / test log-likelihood | Helped |
| Use G-REP instead of ADVI (gamma/beta vs log-normal/logit-normal family) | mean-field VI on sparse models · NIPS perplexity and Olivetti/MNIST/Omniglot test log-likelihood | predictive performance | Helped |
| Use ADVI instead of G-REP (ADVI is faster per iteration) | mean-field VI · Olivetti, NIPS, MNIST, Omniglot | time per iteration: G-REP 1-4x slower than ADVI | Hurt |
| Increase number of Monte Carlo samples from 1 to 2-20 for G-REP (ablation) | G-REP on Olivetti · Olivetti | gradient variance and ELBO | No clear effect |
| Use a sigma chosen so that the correction term g_corr is zero for the sampled z (beta transformation) (ablation) | beta variational distribution in G-REP · beta-gamma MF model | performance on larger-scaled models | Helped |
Closest papers by meaning
- Reparameterization Gradient for Non-differentiable Models (NeurIPS 2018 · similarity 0.93)
- Approximation Based Variance Reduction for Reparameterization Gradients (NeurIPS 2020 · similarity 0.93)
- Reducing Reparameterization Gradient Variance (NeurIPS 2017 · similarity 0.92)
- Implicit Reparameterization Gradients (NeurIPS 2018 · similarity 0.91)
- Sticking the Landing: Simple, Lower-Variance Gradient Estimators for Variational Inference (NeurIPS 2017 · similarity 0.90)
- Rao-Blackwellised Reparameterisation Gradients (NeurIPS 2025 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.