Research map / Probabilistic and scientific modeling / Uncertainty and Gaussian processes
All-in-one simulation-based inference
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces the Simformer, a transformer-based score-based diffusion model trained on the joint of parameters and data that samples arbitrary conditionals (posterior, likelihood, interval-constrained) for simulation-based inference.
Problem
Current amortized simulation-based inference methods are simulation-hungry and inflexible: they need a fixed prior, simulator, and inference task, and cannot handle unstructured/missing data, function-valued parameters, or arbitrary conditionals.
Approach
Train a single score-based diffusion model with a transformer on the joint distribution of parameters and data, using per-variable tokens with condition-state masks and attention masks encoding known dependencies, so that any conditional (posterior, likelihood, arbitrary) can be sampled, with diffusion guidance enabling interval constraints.
Key results
- Across four benchmark tasks, Simformer outperformed NPE in C2ST accuracy (except Gaussian linear with 10k simulations) and required about 10 times fewer simulations on average.
- Simformer accurately models arbitrary conditionals on Two Moons, Tree and HMM tasks as measured by C2ST against MCMC references.
- On Hodgkin-Huxley, interval-constrained guided diffusion yields energy-efficient posteriors whose predictive energy lies below the threshold.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use Simformer instead of NPE for posterior inference | Four benchmark tasks · Four benchmark tasks | C2ST accuracy | Helped |
| Use Simformer with a dense attention mask instead of NPE | Simformer · Gaussian Linear, 10k simulations | C2ST accuracy | Hurt |
| Add domain-knowledge structure to the attention mask | Simformer · Benchmark tasks, particularly Gaussian Linear and SLCP | C2ST accuracy | Helped |
| Use structured attention masks | Simformer · Benchmark tasks and observations | simulation efficiency: about 10 times fewer simulations | Helped |
| Train only on the posterior mask instead of learning all conditional distributions (ablation) | Simformer · Benchmark tasks | C2ST accuracy | No clear effect |
Surprising findings
- Training solely on the posterior mask does not improve performance relative to learning all conditionals.
Limitations
- Sampling requires solving the reverse SDE, slower than normalizing-flow NPE.
- Transformer cost scales quadratically with the number of tokens, challenging training for long sequences.
- Log-probability evaluation requires solving the probability flow ODE, which is computationally costly.
Experimental setup
Datasets: Gaussian Linear Gaussian Mixture Two Moons SLCP Tree HMM Lotka-Volterra simulator SIRD simulator Hodgkin-Huxley simulator
Benchmarks: Gaussian Linear Gaussian Mixture Two Moons SLCP Tree HMM Lotka-Volterra SIRD Hodgkin-Huxley
Metrics: C2ST accuracy expected coverage log-likelihood calibration
Baselines: neural posterior estimation (NPE) neural likelihood estimation (NLE) neural ratio estimation (NRE) MCMC reference posteriors
Builds on: score-based diffusion models (SDE) transformer attention graphically structured diffusion models sbi library
Closest papers by meaning
- Compositional simulation-based inference for time series (ICLR 2025 · similarity 0.90)
- Effortless, Simulation-Efficient Bayesian Inference using Tabular Foundation Models (NeurIPS 2025 · similarity 0.89)
- Leveraging Self-Consistency for Data-Efficient Amortized Bayesian Inference (ICML 2024 · similarity 0.88)
- Variational Inference with Coverage Guarantees in Simulation-Based Inference (ICML 2024 · similarity 0.87)
- Consistency Models for Scalable and Fast Simulation-Based Inference (NeurIPS 2024 · similarity 0.86)
- Learning Likelihood-Free Reference Priors (ICML 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.