Research map / Probabilistic and scientific modeling / Variational and generative sampling
Fisher meets Feynman: score-based variational inference with a product of experts
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a weighted product-of-experts (PoE) variational family built from multivariate t-expert components for score-based black-box VI, using Feynman parameterization for the normalizing constant and importance sampling for sampling.
Problem
Black-box variational inference needs a family that is highly expressive (skew, heavy tails, multimodality) yet tractable to sample from and optimize.
Approach
Each variational member is a weighted product of multivariate t-distribution experts; the normalizing constant is estimated via a Feynman parameterization and samples are drawn by importance sampling over Dirichlet weights, enabling score-based VI on the PoE family.
Key results
- Importance sampler achieves relative ESS above 0.8 on the first two normalizing-constant/sampling examples and roughly 0.3-0.5 on the many-expert example.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use 100 experts with most weights concentrated on 30 active experts and the remaining weights set to 10^{-12} | Importance sampling procedure · Example 3: many sparsely weighted experts | relative ESS: above 0.8 -> roughly between 0.3--0.5 | Hurt |
Surprising findings
- Sufficient integrability condition 2*sum(alpha_k*sign|Lambda_k|) > D was found overly strong; the authors instead used 2*sum(alpha_k) > D and verified integrability.
Limitations
- Importance sampling efficiency drops when many experts with sparse weights are used (relative ESS 0.3-0.5).
- Expert selection is a heuristic that places many experts and reweights them rather than selecting compactly.
Experimental setup
Datasets: posterior-db (garch11, eight-schools, gp-regr) synthetic targets (Gaussian mixture, PoE, diamond, funnel, sinh-arcsinh, Rosenbrock)
Benchmarks: posterior-db synthetic targets (Gaussian mixture, PoE, diamond, funnel, sinh-arcsinh, Rosenbrock)
Metrics: mean absolute error of normalizing constant relative effective sample size forward KL divergence
Baselines: ADVI normalizing flow (realNVP) BaM
Builds on: black-box variational inference realNVP ADVI Adam
Closest papers by meaning
- EigenVI: score-based variational inference with orthogonal function expansions (NeurIPS 2024 · similarity 0.89)
- Score-based Variational Inference via Quantum Maximally Mixed States (NeurIPS 2026 · similarity 0.87)
- Theoretical Guarantees for Variational Inference with Fixed-Variance Mixture of Gaussians (ICML 2024 · similarity 0.87)
- Nearly Dimension-Independent Convergence of Mean-Field Black-Box Variational Inference (NeurIPS 2025 · similarity 0.87)
- Batch and match: black-box variational inference with a score-based divergence (ICML 2024 · similarity 0.86)
- Efficient Mixture Learning in Black-Box Variational Inference (ICML 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.