Research map / Probabilistic and scientific modeling / Uncertainty and Gaussian processes
Multilevel neural simulation-based inference
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes multilevel Monte Carlo training objectives for neural likelihood and posterior estimation (ML-NLE, ML-NPE) that exploit cheap low-fidelity simulators, with variance bounds and optimal sample allocation across levels.
Problem
Neural simulation-based inference needs many simulations for training, which is prohibitive when simulators are expensive; the paper targets reducing this cost when several simulators of varying cost and fidelity are available.
Approach
Rewrites the NLE/NPE training loss as a telescoping multilevel Monte Carlo estimator using seed-matched low- and high-fidelity simulators, allocates samples per level via derived variance bounds, and adds gradient adjustments (norm rescaling and gradient projection) to stabilise optimisation.
Key results
- On the OU process with n1=10 high-fidelity samples, ML-NPE outperforms TL-NPE on both NLPD and KL; with n1=100, TL-NPE slightly outperforms ML-NPE.
- On the toggle-switch model, ML-NLE achieves lower MMD than single-fidelity NLE baselines at matched total computational cost.
- On CAMELS cosmology with 20 high-fidelity and 980 low-fidelity simulations, ML-NPE achieves better NLPD and calibration than NPE trained on 20 high-fidelity simulations; standard NPE is overconfident.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use multilevel training instead of standard MC-loss NLE | NLE on g-and-k · g-and-k | KLD | Helped |
| Use multilevel training instead of standard MC-loss NPE | NPE on g-and-k · g-and-k | NLPD | Helped |
| Use multilevel-NLE instead of cost-matched standard NLE | NLE on toggle-switch · Toggle-switch | MMD | Helped |
| Use ML-NPE instead of TL-NPE with n₁ = 10 | NPE on Ornstein–Uhlenbeck process · Ornstein–Uhlenbeck process, n₁ = 10 | NLPD | Helped |
| Use ML-NPE instead of TL-NPE with n₁ = 10 | NPE on Ornstein–Uhlenbeck process · Ornstein–Uhlenbeck process, n₁ = 10 | KLD | Helped |
| Use ML-NPE instead of TL-NPE with n₁ = 100 | NPE on Ornstein–Uhlenbeck process · Ornstein–Uhlenbeck process, n₁ = 100 | NLPD | Hurt |
| Use ML-NPE instead of TL-NPE with n₁ = 100 | NPE on Ornstein–Uhlenbeck process · Ornstein–Uhlenbeck process, n₁ = 100 | KLD | Hurt |
| Use ML-NPE instead of standard high-fidelity NPE | NPE on cosmological simulations · CAMELS cosmological inference | NLPD | Helped |
| Use ML-NPE instead of standard high-fidelity NPE | NPE on cosmological simulations · CAMELS cosmological inference | Empirical coverage | Helped |
| Add gradient rescaling and gradient projection (ablation) | ML-NLE and ML-NPE optimization · Neural SBI training | Optimization stability | Helped |
| Increase the number of high-fidelity samples | ML-NLE and ML-NPE on g-and-k · g-and-k | KLD (ML-NLE) and NLPD (ML-NPE) | Helped |
| Use ML-NPE instead of low- or high-fidelity-only NPE | NPE on g-and-k · g-and-k | Posterior calibration | Helped |
Surprising findings
- On the OU benchmark with n1=100, the transfer-learning baseline TL-NPE slightly outperforms ML-NPE, and the method shows limitations when high- and low-fidelity simulators have different parameter dimensionality.
Limitations
- Requires log-concave prior and base measure and Sobolev-smoothness assumptions for the theory, which are strong.
- Requires seed-matching (common random numbers) between fidelity levels; not applicable when this is impossible.
- Gradient adjustments add about 15-20% training time and the method struggled when fidelity levels have different parameter dimensions.
Experimental setup
Datasets: g-and-k distribution (simulated) Ornstein-Uhlenbeck process (simulated) toggle-switch model (simulated) CAMELS cosmological simulations
Benchmarks: g-and-k KL divergence OU process NLPD and KL divergence toggle-switch MMD CAMELS NLPD and coverage
Metrics: KL divergence negative log posterior density MMD empirical coverage
Baselines: NLE (MC loss, single fidelity) NPE (MC loss, single fidelity) TL-NPE
Builds on: neural likelihood estimation neural posterior estimation multilevel Monte Carlo transfer learning for NPE (TL-NPE) PCGrad gradient projection
Closest papers by meaning
- Compositional simulation-based inference for time series (ICLR 2025 · similarity 0.90)
- Effortless, Simulation-Efficient Bayesian Inference using Tabular Foundation Models (NeurIPS 2025 · similarity 0.90)
- Simultaneous identification of models and parameters of scientific simulators (ICML 2024 · similarity 0.88)
- Direct Amortized Likelihood Ratio Estimation (AAAI 2024 · similarity 0.86)
- Robust Simulation-Based Inference under Missing Data via Neural Processes (ICLR 2025 · similarity 0.86)
- Simulation-Based Inference with Quantile Regression (ICML 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.