Research map / Probabilistic and scientific modeling / Variational and generative sampling
Automatic Differentiation Variational Inference
Alp Kucukelbir, Dustin Tran, Rajesh Ranganath, Andrew Gelman, David M. Blei
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use transformation T2 = log(exp(theta)-1) instead of log(theta) to map positive variables to the real line (ablation) | ADVI · Gamma(1,2) posterior | KL(q||p): 8.1x10^-2 -> 1.6x10^-2 | Improved |
| use transformation T2 instead of log(theta) (ablation) | ADVI · Gamma(2.5,4.2) posterior | KL(q||p): 3.3x10^-2 -> 3.6x10^-3 | Improved |
| use transformation T2 instead of log(theta) (ablation) | ADVI · Gamma(10,10) posterior | KL(q||p): 8.5x10^-3 -> 7.7x10^-4 | Improved |
| use full-rank instead of mean-field Gaussian approximation (ablation) | ADVI · Stochastic volatility model (500 time steps, simulated) | posterior mean and covariance accuracy | Improved |
| use full-rank instead of mean-field Gaussian approximation (ablation) | ADVI · Two-dimensional Gaussian and logistic regression (simulated) | marginal variance estimates | Improved |
| use full-rank instead of mean-field ADVI on large data (ablation) | ADVI · large datasets | speed | Hurt |
| increase Monte Carlo samples M per gradient estimate above one (ablation) | ADVI · Linear regression with ARD and hierarchical logistic regression | held-out log predictive accuracy over time | No clear effect |
| use ADVI instead of NUTS | Non-negative matrix factorization models · Frey Face (Gamma Poisson, Dirichlet Exponential) | time to held-out predictive accuracy: an order of magnitude speed improvement | Improved |
| use ADVI instead of NUTS and HMC | Gaussian mixture model · imageCLEF subset (1000 images, 10 components) | held-out predictive accuracy | Improved |
| use minibatch size of 500 or larger instead of smaller (ablation) | ADVI with subsampling · imageCLEF full dataset (250,000 images, 30 components) | held-out predictive accuracy: minibatch size 500 or larger reaches high accuracy | Improved |
Closest papers by meaning
- Black Box Variational Inference with a Deterministic Objective: Faster, More Accurate, and Even More Black Box (JMLR 2024 · similarity 0.91)
- Natural Gradient VI: Guarantees for Non-Conjugate Models (NeurIPS 2025 · similarity 0.86)
- EigenVI: score-based variational inference with orthogonal function expansions (NeurIPS 2024 · similarity 0.85)
- Understanding Stochastic Natural Gradient Variational Inference (ICML 2024 · similarity 0.85)
- Denoising Diffusion Variational Inference: Diffusion Models as Expressive Variational Posteriors (AAAI 2025 · similarity 0.84)
- Least squares variational inference (NeurIPS 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
