AI Paper Atlas

Research map / Probabilistic and scientific modeling / Variational and generative sampling

Automatic Differentiation Variational Inference

Alp Kucukelbir, Dustin Tran, Rajesh Ranganath, Andrew Gelman, David M. Blei

JMLR 2017 · Probabilistic and scientific modeling › Variational and generative sampling · approach: Variational inference theory and gradients · 584 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
use transformation T2 = log(exp(theta)-1) instead of log(theta) to map positive variables to the real line (ablation)ADVI · Gamma(1,2) posteriorKL(q||p): 8.1x10^-2 -> 1.6x10^-2Improved
use transformation T2 instead of log(theta) (ablation)ADVI · Gamma(2.5,4.2) posteriorKL(q||p): 3.3x10^-2 -> 3.6x10^-3Improved
use transformation T2 instead of log(theta) (ablation)ADVI · Gamma(10,10) posteriorKL(q||p): 8.5x10^-3 -> 7.7x10^-4Improved
use full-rank instead of mean-field Gaussian approximation (ablation)ADVI · Stochastic volatility model (500 time steps, simulated)posterior mean and covariance accuracyImproved
use full-rank instead of mean-field Gaussian approximation (ablation)ADVI · Two-dimensional Gaussian and logistic regression (simulated)marginal variance estimatesImproved
use full-rank instead of mean-field ADVI on large data (ablation)ADVI · large datasetsspeedHurt
increase Monte Carlo samples M per gradient estimate above one (ablation)ADVI · Linear regression with ARD and hierarchical logistic regressionheld-out log predictive accuracy over timeNo clear effect
use ADVI instead of NUTSNon-negative matrix factorization models · Frey Face (Gamma Poisson, Dirichlet Exponential)time to held-out predictive accuracy: an order of magnitude speed improvementImproved
use ADVI instead of NUTS and HMCGaussian mixture model · imageCLEF subset (1000 images, 10 components)held-out predictive accuracyImproved
use minibatch size of 500 or larger instead of smaller (ablation)ADVI with subsampling · imageCLEF full dataset (250,000 images, 30 components)held-out predictive accuracy: minibatch size 500 or larger reaches high accuracyImproved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy