Research map / Probabilistic and scientific modeling / Uncertainty and Gaussian processes
Bayesian Adaptive Calibration and Optimal Design
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes BACON, a batch-sequential Bayesian adaptive experimental design method that jointly selects simulator designs and calibration parameters by maximising a variational lower bound on expected information gain, using a GP emulator and conditional normalising flows.
Problem
Calibrating expensive computer simulators against real observations: existing approaches fix simulation designs to observed data locations or optimise only calibration parameters, requiring many simulations.
Approach
Treat calibration as Bayesian adaptive experimental design: model simulator and real process jointly as a bi-fidelity GP and maximise a variational EIG lower bound over designs, calibration parameters and a conditional normalising flow posterior, so designs and parameters are optimised jointly.
Key results
- On the 2+2D synthetic unimodal problem, BACON achieved KL(p_T||p_0) of 1.00 ± 0.06 versus 0.89 for IMSPE and 0.62 for random, the highest information gain among methods.
- On the bimodal synthetic posterior, BACON reached KL(p_T||p*) of 0.45 ± 0.06, beating VBMC (0.49 ± 0.13) and other baselines.
- On the soft-robotics calibration with B=16 over T=10, BACON attained KL to reference posterior 1.32 ± 0.05 versus 1.48 to 1.56 for baselines.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use BACON's adaptive information-gain design selection instead of IMSPE | Bayesian simulator calibration · 2+2D synthetic, unimodal posterior | KL(p_T||p_0): 0.89 $\pm$ 0.11 -> 1.00 $\pm$ 0.06 | Helped |
| Use BACON's adaptive information-gain design selection instead of IMSPE | Bayesian simulator calibration · 2+2D synthetic, bimodal posterior | KL(p_T||p*): 0.70 $\pm$ 0.07 -> 0.45 $\pm$ 0.06 | Helped |
| Use BACON instead of VBMC | Bayesian simulator calibration · 2+2D synthetic, unimodal posterior | KL(p_T||p*): 0.53 $\pm$ 0.02 -> 0.76 $\pm$ 0.13 | Hurt |
| Use BACON instead of VBMC | Bayesian simulator calibration · 2+2D synthetic, bimodal posterior | KL(p_T||p*): 0.49 $\pm$ 0.13 -> 0.45 $\pm$ 0.06 | Helped |
| Use BACON instead of IMSPE | Location finding · Location finding | KL(p_T||p*): 0.45 $\pm$ 0.21 -> 0.07 $\pm$ 0.06 | Helped |
| Use BACON instead of IMSPE | Soft-robotics simulator calibration · Soft-robotics simulator calibration | KL(p_T||p*): 1.56 $\pm$ 0.08 -> 1.32 $\pm$ 0.05 | Helped |
| Rely on MAP estimates for design selection with IMSPE rather than flexible posterior approximations | Simulator calibration · Synthetic GP problem, bimodal posterior | posterior approximation quality (KL to p*): 0.45 $\pm$ 0.06 vs 0.70 $\pm$ 0.07 | Hurt |
Surprising findings
- VBMC gave the closest posterior to the full-knowledge posterior in the unimodal synthetic case, outperforming BACON on KL(p_T||p*).
- On the location-finding problem, VBMC's KL to p* was much worse (5.48 ± 1.66) than the GP-based methods.
Limitations
- Cubic computational complexity of exact GP inference limits scalability to large datasets.
- Gaussian assumptions on simulator and discrepancy may be restrictive for heavy-tailed errors.
- Performance may degrade in high-dimensional design/parameter spaces, requiring more iterations.
Experimental setup
Datasets: synthetic GP-sampled simulator location finding hidden sources simulation soft-robotics grasping real measurements (R=10)
Benchmarks: synthetic 2+2D GP problem location finding problem soft-robotics grasping simulator calibration
Metrics: KL divergence to initial posterior KL divergence to reference posterior p* RMSE MAP error
Baselines: Random search IMSPE D-optimal design VBMC
Builds on: Bayesian calibration of computer models (Kennedy and O'Hagan) variational EIG lower bound (Foster et al.) variational Bayesian Monte Carlo conditional normalising flows
Closest papers by meaning
- PASOA- PArticle baSed Bayesian Optimal Adaptive design (ICML 2024 · similarity 0.88)
- On Estimating the Gradient of the Expected Information Gain in Bayesian Experimental Design (AAAI 2024 · similarity 0.88)
- Bayesian Experimental Design Via Contrastive Diffusions (ICLR 2025 · similarity 0.87)
- Approximation-Aware Bayesian Optimization (NeurIPS 2024 · similarity 0.87)
- Online Bayesian Calibration under Gradual and Abrupt System Changes (NeurIPS 2026 · similarity 0.86)
- Optimizing Posterior Samples for Bayesian Optimization via Rootfinding (ICLR 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.