Research map / Probabilistic and scientific modeling / Uncertainty and Gaussian processes
Model-Informed Flows for Bayesian Inference
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Shows that full-rank Gaussian combined with VIP can be exactly written as a generalized forward autoregressive flow, and proposes Model-Informed Flow (MIF) with a translation term, prior-function inputs, and topological ordering for hierarchical Bayesian VI.
Problem
Variational inference struggles with funnel-like posterior geometry in complex hierarchical Bayesian models; the relationship between VIP (learned partial non-centering) and flow-based variational families is unexplored.
Approach
Proves full-rank VIP equals a forward autoregressive flow augmented with an epsilon-dependent translation term and model prior mean/scale as conditioning inputs; builds MIF on this theory, with conditioning networks for shift, log-scale and translation.
Key results
- On German Credit, MIF (h=2^10) reaches -ELBO 519.29 versus 519.39 for FAB and 531.42 for SMC, the best among compared methods.
- On Funnel, MIF achieves -ELBO 0.00 versus 1.86 for mean-field Gaussian, and on Radon it reaches 1211.83 versus 1213.67 for GMMVI.
- Full-rank VIP (FR-VIP) gives lowest -ELBO on all six hierarchical models in the preliminary table, e.g., Radon 1213.52 vs 1220.73 for FR.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use MIF (h=2^10) instead of the strongest listed baseline | Variational inference · German Credit | negative ELBO: 519.35 -> 519.29 | Helped |
| Use MIF (h=2^10) instead of the strongest listed baseline | Variational inference · Funnel | negative ELBO: 0.01 -> 0.00 | Helped |
| Use MIF (h=2^10) instead of the strongest listed baseline | Variational inference · Radon | negative ELBO: 1213.67 -> 1211.83 | Helped |
| Use MIF (h=2^10) instead of the strongest listed baseline | Variational inference · Movielens | negative ELBO: 845.32 -> 842.09 | Helped |
| Use MIF (h=2^10) instead of the strongest listed baseline | Variational inference · IRT | negative ELBO: 815.05 -> 813.17 | Helped |
| Use MIF (h=2^10) instead of the strongest listed baseline | Variational inference · Seeds | negative ELBO: 73.41 -> 72.00 | Helped |
| Use MIF (h=2^10) instead of the strongest listed baseline | Variational inference · Sonar | negative ELBO: 108.59 -> 108.85 | Hurt |
| Use MIF (h=2^10) instead of the strongest listed baseline | Variational inference · Ionosphere | negative ELBO: 111.67 -> 111.85 | Hurt |
| Condition primarily on previous base noise instead of previous latents (ablation) | MIF · Funnel | negative ELBO: 0.01 -> 0.38 | Hurt |
| Remove the translation term (ablation) | MIF · Funnel | negative ELBO: 0.01 -> 0.38 | Hurt |
| Remove explicit prior inputs (ablation) | MIF · Funnel | negative ELBO: 0.01 -> 0.38 | Hurt |
| Remove the hierarchical variable processing order (ablation) | MIF · Funnel | negative ELBO: 0.01 -> 1.86 | Hurt |
Surprising findings
- A single-layer affine MIF without explicit prior inputs performs nearly as well as the full MIF on several models.
- An IAF-like variant conditioned on base noise closes most of the gap to MIF as hidden-unit count grows.
Limitations
- MIF relies on sequential autoregressive generation, which is less parallel than IAFs.
- Wall-clock training time grows substantially with hidden units (e.g., MovieLens 15873 s at 1024 units).
Experimental setup
Datasets: 8Schools German Credit Neal's Funnel Radon Movielens IRT Seeds Sonar Ionosphere
Benchmarks: 8Schools German Credit Funnel Radon Movielens IRT Seeds Sonar Ionosphere
Metrics: negative ELBO
Baselines: Gaussian Mean-Field GMMVI SMC AFT FAB CRAFT UHA FR-VIP IAF MF-VIP
Builds on: Variationally Inferred Parameters inverse autoregressive flow masked autoregressive flow non-centered parameterization
Closest papers by meaning
- Generalized Variational Inference via Optimal Transport (AAAI 2024 · similarity 0.90)
- Inflationary Flows: Calibrated Bayesian Inference with Diffusion-Based Models (NeurIPS 2024 · similarity 0.89)
- Latent Bayesian Optimization via Autoregressive Normalizing Flows (ICLR 2025 · similarity 0.88)
- Variational Inference with Mixtures of Isotropic Gaussians (NeurIPS 2025 · similarity 0.87)
- Learning Likelihood-Free Reference Priors (ICML 2025 · similarity 0.87)
- Streaming Bayes GFlowNets (NeurIPS 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.