Research map / Probabilistic and scientific modeling / Uncertainty and Gaussian processes
ALINE: Joint Amortization for Bayesian Inference and Active Data Acquisition
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes ALINE, a transformer neural-process model trained with policy gradient to jointly perform amortized Bayesian inference and flexible, target-conditioned active data acquisition.
Problem
Existing amortized methods either perform amortized inference with passive data or amortized data acquisition without amortized inference, and acquisition is typically fixed to all parameters or one predictive target, so joint, flexible active inference is lacking.
Approach
A single transformer (TNP-based) with an inference head (Gaussian mixture posteriors/posterior predictives, as in ACE) and an acquisition head is trained with RL, using as per-step reward the improvement in the model's own approximate log-posterior or log-predictive on the chosen target, which is a variational lower bound on sEIG/sEPIG; a target specifier allows runtime switching among parameter s
Key results
- On Location Finding, ALINE achieves EIG lower bound 8.91 ± 0.04 vs. 7.70 for RL-BOED, the best baseline.
- On CES, ALINE reaches 13.50 ± 0.15 EIG lower bound, below RL-BOED's 14.60 but above DAD, vsOED and VPCE.
- On the psychometric task, ALINE is reported as about 10x faster than non-amortized Psi-marginal and QUEST+ comparison methods, with comparable or better inference when targeting guess and lapse rates.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use Aline instead of RL-BOED | Location Finding · Location Finding | EIG lower bound: 7.70 ± 0.06 -> 8.91 ± 0.04 | Helped |
| Use Aline instead of RL-BOED | CES · CES | EIG lower bound: 14.60 ± 0.10 -> 13.50 ± 0.15 | Hurt |
| Use Aline instead of RL-BOED | Location Finding · Location Finding | Training time (h): 63.29 -> 21.20 | Helped |
| Use Aline instead of RL-BOED | CES · CES | Training time (h): 67.28 -> 13.29 | Helped |
| Use Aline instead of RL-BOED | Location Finding · Location Finding | Deployment time (s): 0.0003 ± 0.00 -> 0.03 ± 0.00 | Hurt |
| Use Aline instead of RL-BOED | CES · CES | Deployment time (s): 0.0004 ± 0.00 -> 0.04 ± 0.00 | Hurt |
| Use Aline on unseen benchmark functions instead of baselines | Aline · Out-of-distribution active learning (Higdon, Goldstein-Price, Ackley, Gramacy) | RMSE | Mixed |
| Explicitly target guess and lapse rates rather than all parameters | Aline · Psychometric model | RMSE | Helped |
| Use Aline rather than QUEST+ while targeting threshold and slope | Psychometric model · Threshold and slope targets | RMSE | No clear effect |
Surprising findings
- On CES, ALINE is outperformed by RL-BOED, so it is not best across all BED benchmarks.
- ALINE's deployment time (0.03 s on Location Finding) is higher than MLP-based amortized BED methods due to the transformer.
Limitations
- Uses fixed, pre-defined priors, requiring retraining for different priors.
- Estimates only marginal posteriors, not joint posteriors.
- Architecture tied to fixed input dimensionality and discrete pool-based design spaces.
Experimental setup
Datasets: synthetic GP functions Location Finding simulator CES simulator Higdon Goldstein-Price Ackley Gramacy
Benchmarks: GP active learning benchmarks (Higdon, Goldstein-Price, Ackley, Gramacy) Location Finding Constant Elasticity of Substitution psychometric model
Metrics: RMSE EIG lower bound (sPCE) log probability of true parameter deployment time training time
Baselines: GP-US GP-VR GP-EPIG GP-RS ACE-US Random design VPCE DAD vsOED RL-BOED QUEST+ Psi-marginal
Builds on: ACE Transformer Neural Processes DAD EPIG sEIG variational bounds
Closest papers by meaning
- Efficient Adaptive Data Acquisition via Pretrained Belief Representations (NeurIPS 2026 · similarity 0.87)
- Amortized Active Causal Induction with Deep Reinforcement Learning (NeurIPS 2024 · similarity 0.86)
- Amortized Bayesian Experimental Design for Decision-Making (NeurIPS 2024 · similarity 0.86)
- All-in-one simulation-based inference (ICML 2024 · similarity 0.86)
- Efficient Biological Data Acquisition through Inference Set Design (ICLR 2025 · similarity 0.86)
- Adaptive teachers for amortized samplers (ICLR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.