Research map / Probabilistic and scientific modeling / Variational and generative sampling
Evaluating the Variance of Likelihood-Ratio Gradient Estimators
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use optimal RAM estimator (fully-informed baseline) instead of plain likelihood-ratio estimator for variational learning of SBN | Sigmoid belief network variational learning (2-layer and 4-layer) · MNIST and Omniglot | gradient variance | Helped |
| Add constant baseline (moving average of f) to likelihood-ratio estimator (ablation) | LR estimator on SBN · MNIST and Omniglot, 2-layer and 4-layer SBN | gradient variance | Helped |
| Add input-dependent baseline (IDB) on top of constant baseline (ablation) | LR+C estimator on SBN · MNIST and Omniglot, 2-layer and 4-layer SBN | gradient variance | Helped |
| Use local expectation gradient (LEG) rather than constant baseline LR (ablation) | LEG estimator on SBN · MNIST and Omniglot, 2-layer and 4-layer SBN | gradient variance | No clear effect |
| Gap between practical baselines and optimal estimator remains | Modern baselines (LR+C+IDB, MuProp+C+IDB) on SBN · MNIST and Omniglot | gradient variance | Mixed |
| Compute RAM estimator with GPU parallelization versus plain LR | RAM estimator · SBN training cost | computational time relative to LR: two times larger than LR | No clear effect |
Closest papers by meaning
- Reparameterization Gradient for Non-differentiable Models (NeurIPS 2018 · similarity 0.90)
- Reducing Reparameterization Gradient Variance (NeurIPS 2017 · similarity 0.89)
- Sticking the Landing: Simple, Lower-Variance Gradient Estimators for Variational Inference (NeurIPS 2017 · similarity 0.89)
- The Generalized Reparameterization Gradient (NeurIPS 2016 · similarity 0.89)
- Understanding Stochastic Natural Gradient Variational Inference (ICML 2024 · similarity 0.88)
- Rao-Blackwellised Reparameterisation Gradients (NeurIPS 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.