Research map / Probabilistic and scientific modeling / Uncertainty and Gaussian processes
Effortless, Simulation-Efficient Bayesian Inference using Tabular Foundation Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Repurposes the tabular foundation model TabPFN as a training-free autoregressive posterior estimator for simulation-based inference (NPE-PFN, TSNPE-PFN), improving simulation efficiency on benchmarks and scientific simulators.
Problem
Simulation-based inference typically needs large numbers of simulations and training/tuning of an inference network, which is costly for expensive simulators and burdensome for non-experts.
Approach
Use TabPFN as a pre-trained, in-context conditional density estimator applied autoregressively over parameter dimensions, so that posterior inference requires no network training; add relevance-based filtering of simulations to exceed TabPFN's context limit, and a ratio-trick classifier for fast density evaluation in truncated sequential rounds.
Key results
- On the SBI benchmark at 10^2 simulations, NPE-PFN substantially outperforms NLE and NPE on all but one task.
- TSNPE-PFN reaches close to optimal C2ST on SIR with only 10^2 simulations.
- On the pyloric network, TSNPE-PFN reaches over 50% valid simulations with 5*10^3 simulations and about 99% validity below 5*10^4 simulations.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use NPE-PFN instead of flow-based NPE at 10^2 simulations | amortized SBI benchmark tasks · SBI benchmark tasks | C2ST accuracy | Helped |
| Use NPE-PFN instead of NPE on SIR and Lotka-Volterra | amortized SBI · SIR and Lotka-Volterra | C2ST accuracy | Helped |
| Use NLE instead of NPE-PFN | amortized SBI · SLCP | C2ST accuracy | Hurt |
| Filter simulations by relevance when using 10^5 simulations (ablation) | NPE-PFN · SBI benchmark tasks | C2ST accuracy | Helped |
| Extend context without filtering to 10^5 simulations (ablation) | NPE-PFN · SBI benchmark tasks | C2ST accuracy | No clear effect |
| Use NPE-PFN instead of NPE under prior or likelihood misspecification | amortized SBI · 2D Gaussian means misspecification | C2ST accuracy | Helped |
| Use TSNPE-PFN instead of TSNPE | sequential SBI · Single-Compartment Hodgkin-Huxley Model | predictive accuracy | Helped |
| Use TSNPE-PFN instead of TSNPE | sequential SBI · Allen cell type observations | posterior predictive match | Helped |
| Use TSNPE-PFN with restricted prior and importance resampling | sequential SBI · Pyloric Network | valid simulations: 96.83% | Helped |
| Use TSNPE-PFN instead of prior comparison methods | sequential SBI · Pyloric Network | valid simulations: 96.83% | Helped |
| Vary the autoregressive parameter order (ablation) | TSNPE-PFN · Pyloric Network | energy score: 0.081 - 0.094 | No clear effect |
| Vary the autoregressive parameter order (ablation) | NPE-PFN · SBI benchmark tasks | C2ST accuracy | No clear effect |
Surprising findings
- NPE-PFN is orders of magnitude slower at inference than NPE, comparable to NLE.
- Unfiltered NPE-PFN with 10^5 simulations does not improve over the default and filtering is needed to match baselines.
Limitations
- TabPFN feature limit (~500) and context limit (~10^4) constrain high-dimensional parameters and observations.
- Advantage over standard NPE diminishes as simulation budgets and density-evaluation needs grow.
- Autoregressive evaluation makes inference more expensive than flow-based NPE.
Experimental setup
Datasets: SBI benchmark tasks (Gaussian Linear, Gaussian Mixture, Two Moons, SLCP, Bernoulli GLM, SIR, Lotka-Volterra) Allen cell type database observations UCI datasets Pyloric network simulations
Benchmarks: SBI benchmark Hodgkin-Huxley (Allen cell types) pyloric network misspecified 2D Gaussian means
Metrics: C2ST simulation-based calibration runtime average distance to observation percentage of valid simulations energy score
Baselines: NPE NLE NRE TSNPE SNLE NPE ensembles SNVI S-UNLE
Builds on: TabPFN Neural Posterior Estimation Truncated Sequential NPE ABC-style filtering ratio trick
Closest papers by meaning
- Compositional simulation-based inference for time series (ICLR 2025 · similarity 0.90)
- Multilevel neural simulation-based inference (NeurIPS 2025 · similarity 0.90)
- Simultaneous identification of models and parameters of scientific simulators (ICML 2024 · similarity 0.89)
- Consistency Models for Scalable and Fast Simulation-Based Inference (NeurIPS 2024 · similarity 0.89)
- All-in-one simulation-based inference (ICML 2024 · similarity 0.89)
- Learning Likelihood-Free Reference Priors (ICML 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.