Research map / Probabilistic and scientific modeling / Uncertainty and Gaussian processes
Leveraging Self-Consistency for Data-Efficient Amortized Bayesian Inference
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a self-consistency loss that penalizes variance of the log marginal likelihood estimate across parameter samples, added to NPE/NPLE training to improve amortized Bayesian inference in low-data regimes.
Problem
Amortized Bayesian inference with neural posterior (NPE) and neural posterior-likelihood (NPLE) estimators often needs large simulation budgets and does not explicitly enforce accurate marginal likelihood estimates.
Approach
Invert Bayes' theorem so the marginal likelihood p(Y)=p(θ)p(Y|θ)/p(θ|Y) must be constant across θ; penalize the variance of its log estimate across prior/proposal samples (proposal = approximate posterior) as an extra training loss, with annealed weight.
Key results
- On Two Moons, SC-NPLE at N=512 matches NPLE trained on N=4096 in posterior MMD, per the paper's statement.
- On the Two Moons LML estimate, SC-NPLE's 95% CI width is 1.14 vs 10.21 for NPLE at N=4096.
- In the high-dimensional time series model (N=2048), MMD drops from 0.88 (NPLE) to 0.54 with SC-NPLE using K=100.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add self-consistency loss to NPLE | NPLE · Two Moons (N=1024) | log likelihood density: 3.18 ± 0.03 -> 3.45 ± 0.02 | Helped |
| Add self-consistency loss to NPLE | NPLE · Two Moons (N=2048) | LML approximation error (95% CI width): 9.07 ± 0.06 -> 1.21 ± 0.01 | Helped |
| Add self-consistency loss to NPLE | NPLE · Two Moons (N=512) | log likelihood density: 3.15 ± 0.03 -> 3.14 ± 0.02 | No clear effect |
| Add self-consistency loss to NPE | NPE · Source Location Finding (N=512) | MMD: 0.54 ± 0.02 -> 0.40 ± 0.03 | Helped |
| Add self-consistency loss to NPLE | NPLE · High-dimensional time series (N=2048) | MMD: 0.88 ± 0.06 -> 0.54 ± 0.03 | Helped |
| Increase consistency samples from K=5 to K=100 (ablation) | SC-NPLE · High-dimensional time series (N=2048) | MMD: 0.78 ± 0.04 -> 0.54 ± 0.03 | Helped |
| Increase consistency samples beyond K=10 (ablation) | SC-NPE · Gaussian mixture model | MMD | No clear effect |
| Vary consistency sample count K (ablation) | SC-NPE · Source Location Finding | MMD | No clear effect |
| Use an underexpressive linear likelihood network (ablation) | SC-NPLE · Hes1 Expression | posterior MMD | Hurt |
| Add self-consistency loss using an approximate likelihood | NPLE · Hes1 Expression | simulation-based calibration | Helped |
| Add self-consistency loss to NPE | NPE · Gaussian mixture model | MMD | Helped |
| Add self-consistency loss to NPLE | NPLE · Two Moons | posterior MMD | Helped |
Surprising findings
- Increasing the number of self-consistency Monte Carlo samples K beyond 10 did not noticeably improve performance in the Gaussian mixture and source-location experiments.
Limitations
- Requires evaluating the prior density p(θ) analytically or learning it.
- Adds computational cost during training due to K Monte Carlo samples per step.
- Using an approximate likelihood for self-consistency makes the variance estimate noisier; an underexpressive likelihood network gives larger MMD for SC-NPLE.
Experimental setup
Datasets: simulated Gaussian mixture data two moons simulator Hes1 simulator with experimental data from Silk et al. source location finding simulator autoregressive compartmental time series simulator
Benchmarks: Gaussian mixture model Two Moons Hes1 expression model source location finding high-dimensional compartmental time series
Metrics: maximum mean discrepancy simulation-based calibration log likelihood density of observed data LML estimate CI width
Baselines: NPE NPLE SNPE
Builds on: neural posterior estimation neural posterior and likelihood estimation neural spline flows sequential neural posterior estimation simulation-based calibration
Closest papers by meaning
- Amortized Simulation-Based Inference in Generalized Bayes via Neural Posterior Estimation (ICML 2026 · similarity 0.90)
- Robust Amortized Bayesian Inference with Self-Consistency Losses on Unlabeled Data (ICLR 2026 · similarity 0.89)
- Robust Amortized Simulation-Based Inference via Learned Error Models (NeurIPS 2026 · similarity 0.89)
- All-in-one simulation-based inference (ICML 2024 · similarity 0.88)
- Variational Inference with Coverage Guarantees in Simulation-Based Inference (ICML 2024 · similarity 0.86)
- Compositional simulation-based inference for time series (ICLR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.