Research map / Probabilistic and scientific modeling / Variational and generative sampling
Variational inference theory and gradients
Gives guarantees and lower-variance gradient estimators for variational inference.
89 accepted papers at ICML, NeurIPS, ICLR, CVPR and AAAI (2016–2026) · an approach in Variational and generative sampling, keyword cluster “variational inference · vi · lower”
State History Tried Checklist Preprints Who Export
Where it stands
Not enough benchmark data Fewer than four papers report the same benchmark, so there is no record to follow.
17 papers in 2023–24 and 16 in 2025–26 (at ICML, NeurIPS, ICLR, CVPR and AAAI); its share of the topic shrank, from 4.1% to 3.2%.
How this is decided
A benchmark is tracked when at least four papers of this approach report their own method on the same dataset and metric in at least two different years. Its record is the best number reported up to 2024 compared with the best reported in 2025–26: still improving means at least 1% better, flat means at most 0.2% better with at least two papers from 2025–26 trying. Numbers on a different scale from the rest (a fraction among percentages) and far outliers are left out. Results come from different papers with different setups, so they are signals, not head-to-head comparisons. The share of papers is counted at ICML, NeurIPS, ICLR, CVPR and AAAI.
How it got here
Most cited paper of each year
- 2016 · Variational Inference for Monte Carlo Objectives (ICML 2016 · 141 citations)
- 2017 · Hierarchical Implicit Models and Likelihood-Free Variational Inference (NeurIPS 2017 · 96 citations)
- 2018 · Yes, but Did It Work?: Evaluating Variational Inference (ICML 2018 · 84 citations)
- 2019 · Understanding and Accelerating Particle-Based Variational Inference (ICML 2019 · 41 citations)
- 2020 · Decision-Making with Auto-Encoding Variational Bayes (NeurIPS 2020 · 23 citations)
- 2021 · Functional Variational Inference based on Stochastic Process Generators (NeurIPS 2021 · 2 citations)
- 2022 · Variational methods for simulation-based inference (ICLR 2022 · 10 citations)
- 2023 · Forward-Backward Gaussian Variational Inference via JKO in the Bures-Wasserstein Space (ICML 2023 · 1 citation)
- 2024 · Generalized Variational Inference via Optimal Transport (AAAI 2024 · 1 citation)
- 2025 · Denoising Diffusion Variational Inference: Diffusion Models as Expressive Variational Posteriors (AAAI 2025 · 2 citations)
- 2026 · Likelihood-free inference of phylogenetic tree posterior distributions (NeurIPS 2026 · 0 citations)
Earliest papers here
- Variational Inference for Monte Carlo Objectives (ICML 2016 · 141 citations)
- Rényi Divergence Variational Inference (NeurIPS 2016 · 70 citations)
- Operator Variational Inference (NeurIPS 2016 · 57 citations)
What has been tried
77 papers here report 714 measured changes (a component added, removed or swapped, compared with a baseline): helped 69%, no clear effect 10%, hurt 17%, mixed 4%. Papers mostly publish changes that helped, so "helped" is over-represented.
Ablations in recent papers
| What was changed | Measured on | Result | Effect | Paper |
|---|---|---|---|---|
| add the sleep term | MNIST, pinwheel prior · Latent NLL | 2.12 -> 1.27 ± 0.21 | Helped | Denoising Diffusion Variational Inference: Diffusion Models as Expressive Variational Posteriors (AAAI 2025 · 2 citations) |
| Use the scheduled λ instead of fixing λ=1 | concrete · RMSE | 5.1378 -> 4.9419 | Helped | Generalized Variational Inference via Optimal Transport (AAAI 2024 · 1 citation) |
| optimize the forward-KL SAA with L-BFGS and line search | Forward-KL variational inference · optimization stability | Hurt | VISA: Variational Inference with Sequential Sample-Average Approximations (NeurIPS 2024 · 1 citation) | |
| Use AISIVI instead of BSIVI (importance sampling rather than the base score estimator) | Banana · KL(p, q) | 0.3022 -> 0.0853 | Helped | Revisiting Unbiased Implicit Variational Inference (ICML 2025 · 0 citations) |
| replace posterior variances with fractional-posterior variances while retaining ELBO means | GMM component means · coverage | Lγ γ=0.1: 1.0000, 1.0000; C0.1: 1.0000, 1.0000 | No clear effect | Variational Learning of Fractional Posteriors (ICML 2025 · 0 citations) |
| Add importance sampling (IS) | Simulated datasets SD2–SD5 · E-KL | 1.66 -> 1.64 (SD2); 2.38 -> 2.23 (SD3); 2.88 -> 2.64 (SD4); 4.45 -> 4.11 (SD5) | Helped | Censor Dependent Variational Inference (ICML 2025 · 0 citations) |
| Use SoftCVI with α=0.75 instead of α=1 | Bayesian neural additive model · test log-likelihood | -2.424 ± 0.007 vs -2.422 ± 0.007; -2.423 ± 0.005 vs -2.423 ± 0.005 | No clear effect | SoftCVI: Contrastive variational inference with self-generated soft labels (ICLR 2025 · 0 citations) |
| Use score-function rather than pathwise gradients to minimize forward χ² divergence | Toy mixture model · forward χ² divergence minimization performance | Helped | Forward $\chi^2$ Divergence Based Variational Importance Sampling (ICLR 2024 · 0 citations) | |
| Increase the number of mixture components A while holding S fixed in S2S | MNIST · NLL | Helped | Efficient Mixture Learning in Black-Box Variational Inference (ICML 2024 · 0 citations) | |
| allow non-fixed mixture weights | Gaussian mixture approximation · reverse KL rate | 1/n | Helped | Theoretical Guarantees for Variational Inference with Fixed-Variance Mixture of Gaussians (ICML 2024 · 0 citations) |
| Replace Gaussian kernel with Reisz kernel | Product of two Student’s t-distributions, edge width 5 · Wasserstein distance | 0.1048 ± 0.03 -> 0.0451 ± 0.01 | Helped | Kernel Semi-Implicit Variational Inference (ICML 2024 · 0 citations) |
| Replace the mean-field posterior with TreeVI | Synthetic VAE · ELBO | -11.1535 -> -10.8998 | Helped | TreeVI: Reparameterizable Tree-structured Variational Inference for Instance-level Correlation Capturing (NeurIPS 2024 · 0 citations) |
What reviewers will expect
Datasets and baselines named in the 14 papers of this approach from 2025–2026 that have a full-text summary. Leaving out the common ones is a frequent review question.
Datasets
- MNIST · 2 of 14 papers
Baselines
- ADVI · 3 of 14 papers
- ELBO · 2 of 14 papers
- BaM · 2 of 14 papers
Recent preprints
Loading recent arXiv preprints…
Who publishes on it
Institutions on the most papers of this approach since 2023 (of 33 papers; from OpenAlex and the papers' affiliations).
- Flatiron Institute · 4 papers
- Columbia University · 4 papers
- Peking University · 3 papers
- University of California, San Diego · 3 papers
- Korea Advanced Institute of Science and Technology · 2 papers
- University of Edinburgh · 2 papers
- Simons Foundation · 2 papers
- Sun Yat-sen University · 2 papers
Use it
Download BibTeX of 40 key papersOpen the topic in the interactive map
Get alerts. Describe your idea in scime atlas and save it (free scime account): you get a weekly email when new preprints land close to it.
Related approaches
Same keyword cluster
- Annealing and divergence-based variational objectives (79 papers)
- Amortized and online variational inference (59 papers)
- Variational methods for sequence models (40 papers)
Elsewhere in the topic
- Likelihood-free and graphical model estimation (161 papers)
- Flow matching and flow sampling (144 papers)
- Posterior sampling with diffusion guidance (137 papers)
- Score-based generative modeling analysis (128 papers)
- Structured latent VAEs (125 papers)
- Exact inference for probabilistic programs (103 papers)
- Disentangled and discrete latent models (92 papers)
- Latent variable marginalization and estimators (91 papers)
Benchmark numbers, ablations and baseline numbers are extracted by AI from each paper's full text and can be wrong; the paper is the authority. Data: results index of 2026-11-02, citation counts from OpenAlex (2026-10-09).