Research map / Probabilistic and scientific modeling / Uncertainty and Gaussian processes
Learning Likelihood-Free Reference Priors
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes likelihood-free methods (generative entropy difference and variational MI lower bounds with InfoNCE/SMILE) that learn variational reference priors for simulation models using only simulator samples.
Problem
Constructing minimally informative reference priors for simulation models with intractable likelihoods, where the mutual information between parameters and data cannot be computed directly.
Approach
Learn a normalizing-flow variational prior by maximizing a likelihood-free estimate of the mutual information between parameters and simulated data, either via a difference of entropy estimates (GED with a learned conditional posterior density) or via variational MI lower bounds (InfoNCE, SMILE) with surrogate gradients for differentiable simulators; learned critics or posteriors then support SBI
Key results
- On tractable models, GED, InfoNCE and SMILE all outperform the numerical method of Berger et al. (2009) on Wasserstein and C2ST for the Exponential, Triangular and AR(1) models; InfoNCE is slightly worse than the baseline on the Gaussian scale model in Wasserstein distance.
- On SLCP-D, C2ST accuracy for learned priors reaches 0.99 (InfoNCE) and 0.98 (SMILE) versus 0.55 for the Uniform prior and 0.69 for GED.
- On the SIR agent-based model, trajectories from InfoNCE and GED priors are more diverse than from a uniform prior, and posterior predictive samples from GED/InfoNCE-derived inference match NPE/NRE results.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use GED to learn the reference prior instead of the numerical method | Gaussian Scale · Gaussian Scale | Wasserstein: 6.95 -> 1.66 | Helped |
| Use InfoNCE to learn the reference prior instead of the numerical method | Exponential · Exponential | Wasserstein: 4.25 -> 2.43 | Helped |
| Use GED to learn the reference prior instead of the numerical method | Triangular · Triangular | Wasserstein: 0.08 -> 0.02 | Helped |
| Use SMILE to learn the reference prior instead of the numerical method | AR(1) · AR(1) | Wasserstein: 0.14 -> 0.05 | Helped |
| Use GED to learn the reference prior instead of the numerical method | Gaussian Scale · Gaussian Scale | C2ST: 0.90 -> 0.49 | Helped |
| Use InfoNCE to learn the reference prior instead of the numerical method | Exponential · Exponential | C2ST: 0.91 -> 0.58 | Helped |
| Use GED to learn the reference prior instead of the numerical method | Triangular · Triangular | C2ST: 0.64 -> 0.51 | Helped |
| Use InfoNCE to learn the reference prior instead of the numerical method | AR(1) · AR(1) | C2ST: 0.62 -> 0.50 | Helped |
| Learn an SLCP-D prior with InfoNCE instead of using a Uniform prior | SLCP-D · SLCP-D | C2ST: 0.55 -> 0.99 | Helped |
| Learn an SLCP-D prior with SMILE instead of using a Uniform prior | SLCP-D · SLCP-D | C2ST: 0.55 -> 0.98 | Helped |
| Learn an SLCP-D prior with GED instead of using a Uniform prior | SLCP-D · SLCP-D | C2ST: 0.55 -> 0.69 | Helped |
Surprising findings
- GED, though producing reasonable reference priors, is less accurate than the variational lower bound methods on the SLCP-D model with complex posteriors.
- Learned reference priors are harder to estimate than expected: multiple runs of the same method can give substantially different priors, indicating multiple local optima.
Limitations
- Only handles settings where all parameters are of interest; nuisance parameters not treated.
- Learned priors are continuous proper approximations to reference priors; asymptotes are hard to reproduce and RPs are not necessarily proper.
- Currently VLB methods require differentiable simulators (surrogate gradients used for non-differentiable ones); GED is sensitive to learning rates and can be unstable.
Experimental setup
Datasets: Gaussian scale model (simulated) Exponential rate model (simulated) Triangular model (simulated) AR(1) model (simulated) SLCP-D (simulated) SIR agent-based model (simulated)
Benchmarks: Gaussian scale model Exponential rate model Triangular model AR(1) model SLCP-D SIR agent-based model
Metrics: Wasserstein distance classifier two-sample test (C2ST) accuracy
Baselines: Berger et al. (2009) numerical method Uniform prior NPE NRE
Builds on: Berger et al. 2009 reference prior algorithm Pichler et al. 2022 Jarne Ornia et al. 2024 InfoNCE SMILE MINE
Closest papers by meaning
- Consistency Models for Scalable and Fast Simulation-Based Inference (NeurIPS 2024 · similarity 0.89)
- Effortless, Simulation-Efficient Bayesian Inference using Tabular Foundation Models (NeurIPS 2025 · similarity 0.88)
- Simultaneous identification of models and parameters of scientific simulators (ICML 2024 · similarity 0.87)
- Addressing Misspecification in Simulation-based Inference through Data-driven Calibration (ICML 2025 · similarity 0.87)
- Model-Informed Flows for Bayesian Inference (NeurIPS 2025 · similarity 0.87)
- Generalized Variational Inference via Optimal Transport (AAAI 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.