Research map / Probabilistic and scientific modeling / Variational and generative sampling
Variational Learning of Fractional Posteriors
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces a one-parameter variational lower bound on log-evidence (via Hölder's inequality) whose optimum yields approximate fractional posteriors, with analytic updates for exponential-family and mixture models and VAE experiments.
Problem
Standard ELBO-based variational inference tends to give overconfident posteriors and poor calibration under misspecification, and offers no flexible tempering between evidence maximisation and posterior calibration.
Approach
Derives a one-parameter lower bound on log-evidence using Hölder's inequality whose optimum is a fractional posterior p(D|z)^gamma p(z); extends it to hierarchical (mixing) fractional posteriors and to joint Bayes and fractional posteriors, reducing to ELBO as gamma goes to 1.
Key results
- On the 2-component GMM calibration study, fractional posteriors with gamma 0.1 to 0.5 give conservative coverage (about 1.0 to 0.988 at alpha 0.05), while ELBO (gamma=1) gives 0.928 and 0.918 for the two means.
- On Fashion-MNIST, decoder FID for generation from the prior improves from 83.5 (gamma=1.0) to 67.8 (gamma=1e-3), and 68.8 at gamma=1e-5.
- On VAE MNIST, the gamma=0.1 bound reaches a test-set objective of 1677.2 versus 1583.2 for ELBO with explicit posterior.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use fractional posterior with γ≤0.8 instead of ELBO | GMM inference · GMM component means | coverage calibration: ELBO: 0.9278, 0.9182; Lγ γ=0.1: 1.0000, 1.0000 | Helped |
| decrease γ from 1.0 to 0.1 | VAE with explicit posterior · MNIST test | log-evidence bound: 1583.2 sd{14.6} -> 1677.2 sd{3.4~} | Helped |
| use a semi-implicit rather than explicit posterior at γ=0.1 | VAE · MNIST test | log-evidence bound: 1677.2 sd{3.4~} -> 1678.7 sd{3.1} | Helped |
| jointly learn Bayes and fractional posteriors with LBbhγ rather than LBhγ | VAE with semi-implicit posterior · MNIST | log-evidence bound: null | Hurt |
| train the decoder with a fractional posterior instead of the Bayes posterior | VAE decoder · Fashion-MNIST test set | FID: 83.5 -> 68.8 | Helped |
| use β-VAE instead of the proposed fractional-posterior objective for very small γ | VAE decoder · Fashion-MNIST test set | FID: 67.8 -> 334.7; 68.8 -> 342.3 | Hurt |
| replace posterior variances with fractional-posterior variances while retaining ELBO means (ablation) | GMM calibration · GMM component means | coverage: Lγ γ=0.1: 1.0000, 1.0000; C0.1: 1.0000, 1.0000 | No clear effect |
| calibrate γ using interval lengths | GMM fractional posterior · GMM component means | coverage: Rℓ: 0.9588, 0.9474 | Helped |
| calibrate γ using empirical coverages | GMM fractional posterior · GMM component means | coverage: Rκ: 0.9570, 0.9458 | Helped |
| decrease γ from 1.0 to 0.1 (ablation) | VAE with explicit posterior · MNIST test | log-evidence bound: 1583.2 sd{14.6} -> 1677.2 sd{3.4~} | Helped |
| use a semi-implicit instead of explicit posterior (ablation) | VAE with γ=0.1 · MNIST test | log-evidence bound: 1677.2 sd{3.4~} -> 1678.7 sd{3.1} | Helped |
Surprising findings
- The Bayes-posterior-plus-fractional joint bound (LBbh) does not outperform the semi-implicit fractional bound LBh_gamma despite having more parameters.
- The beta-VAE objective becomes unstable at small gamma-equivalent settings, with FID rising to 334.7 and 342.3 at gamma 1e-3 and 1e-5.
Limitations
- Mixture-model bound requires an ELBO step for cluster assignments, so the fully fractional approach is not closed-form.
- Renyi divergence is finite in fewer cases than KL, which can complicate optimisation.
- Monte Carlo estimates need more than one sample to be effective, which may be unrealistic for huge datasets.
Experimental setup
Datasets: MNIST Fashion-MNIST synthetic Gaussian mixture data
Benchmarks: MNIST Fashion-MNIST
Metrics: empirical coverage average interval length log-evidence bound Frechet inception distance
Baselines: ELBO variational Rényi bound beta-VAE objective semi-implicit VI (LBh_1.0)
Builds on: variational Rényi bound ELBO generalized variational inference beta-VAE semi-implicit variational inference
Closest papers by meaning
- Recursive Inference for Variational Autoencoders (NeurIPS 2020 · similarity 0.90)
- Hierarchical Decompositional Mixtures of Variational Autoencoders (ICML 2019 · similarity 0.90)
- Variational Boosting: Iteratively Refining Posterior Approximations (ICML 2017 · similarity 0.89)
- Beyond Vanilla Variational Autoencoders: Detecting Posterior Collapse in Conditional and Hierarchical Variational Autoencoders (ICLR 2024 · similarity 0.89)
- Variational Inference with Mixtures of Isotropic Gaussians (NeurIPS 2025 · similarity 0.89)
- VIKING: Deep variational inference with stochastic projections (NeurIPS 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.