Research map / Probabilistic and scientific modeling / Variational and generative sampling

Stochastic variance-reduced Gaussian variational inference on the Bures-Wasserstein manifold

ICLR 2025 · Probabilistic and scientific modeling › Variational and generative sampling · approach: Annealing and divergence-based variational objectives · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes SVRGVI, a control-variate variance-reduced gradient estimator for Gaussian variational inference on the Bures-Wasserstein manifold that adds no extra samples and proves variance reduction and improved convergence bounds over SGVI.

Problem

Single-sample Monte Carlo estimates of the Bures-Wasserstein gradient of the potential energy in Gaussian variational inference are too noisy, causing slow convergence and poor accuracy, especially in high dimensions.

Approach

Uses the Stein/Hyvärinen score of the current Gaussian approximation, Σ_k^{-1}(X_k − m_k), as a zero-mean control variate for the single-sample gradient estimate ∇V(X_k), with the Hessian estimator left unchanged; the Cholesky factor used for sampling is reused so overhead is O(d^2).

Key results

• On Gaussian targets at d=200, SVRGVI reaches KL around 10^-2 while SGVI and BWGD stay around 10^3, a five-order-of-magnitude gap.
• Empirically the control variate reduces gradient estimator variance by roughly a factor of 10 at the optimal c around 1.
• Theory shows the estimator has smaller variance than the Monte Carlo estimator in a neighbourhood of the optimum and at every Gaussian with sufficiently large variance for strongly convex V, and improves the SGVI convergence bounds.

Measured effects reported in the paper

Surprising findings

• The Monte Carlo BWGD and SGVI are effectively identical except that BWGD is unstable, and their performance degrades in high dimensions.
• BWGD performs worse on average than the Laplace approximation on the Student's t target.

Limitations

• Restricted to Gaussian approximations by construction
• Cubic computational cost due to Hessian computation
• Experiments are on synthetic targets and one logistic regression model

Experimental setup

Benchmarks: Gaussian targets (d=10,50,200) Student's t target (d=200) Bayesian logistic regression (n=1000, d=200)

Metrics: KL divergence objective F estimate empirical variance of gradient estimator

Baselines: BWGD SGVI EVI Laplace approximation minibatching

Builds on: SGVI (Diao et al. 2023 Bures-Wasserstein stochastic gradient) forward-backward Euler Särkkä variational Kalman filtering ODEs Stein's lemma

Closest papers by meaning

• Forward-Backward Gaussian Variational Inference via JKO in the Bures-Wasserstein Space (ICML 2023 · similarity 0.92)
• Variational inference via Wasserstein gradient flows (NeurIPS 2022 · similarity 0.90)
• Theoretical Guarantees for Variational Inference with Fixed-Variance Mixture of Gaussians (ICML 2024 · similarity 0.89)
• Asymptotics of Alpha-Divergence Variational Inference Algorithms with Exponential Families (NeurIPS 2024 · similarity 0.88)
• Least squares variational inference (NeurIPS 2025 · similarity 0.88)
• Natural Gradient VI: Guarantees for Non-Conjugate Models (NeurIPS 2025 · similarity 0.87)