Research map / Probabilistic and scientific modeling / Bayesian optimization
$\alpha$-PFN: Fast Entropy Search via In-Context Learning
Herilalaina Rakotoarison, Steven Adriaensen, Tom Julian Viering, Carl Hvarfner, Samuel Müller, Frank Hutter, Eytan Bakshy
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace MC-based GP entropy search (MES) with an alpha-PFN that predicts the acquisition in one forward pass | MES · HPO-B-7609 (9D) | speedup: 74.5 -> 1.1 minutes (65.0x) | Improved |
| replace MC-based GP entropy search with alpha-PFN across variants | JES, MES, PES · synthetic, LCBench and HPO-B tasks | runtime: speedups from 1.6x to over 72x | Improved |
| alpha-PFN approximation of PES, MES and JES in place of GP-MCMC versions | entropy search acquisition · synthetic functions, inference regret | inference regret | No clear effect |
| alpha-PFN JES versus GP baselines on real HPO tasks | JES · LCBench and HPO-B | predicted best accuracy / average rank | Improved |
| alpha-PFN MES versus GP MES | MES · HPO-B | average rank of inference regret | Hurt |
| alpha-PFN variants versus GP on Hartmann | JES, MES · Hartmann 4D vs 6D | inference regret | Mixed |
| train with clustered BO-like traces instead of uniformly sampled context and query points (ablation) | alpha-PFN · higher-dimensional synthetic functions | inference regret | Improved |
| increase observation noise to 0.5 (out of distribution) at test time (ablation) | alpha-PFN and GP baselines · Hartmann 4D and 6D | inference regret: sigma_n 0.316 -> 0.5 | Hurt |
| train a second-stage alpha-PFN on information gains instead of Monte Carlo averaging of base-PFN at inference (ablation) | alpha-PFN · 1D JES qualitative comparison | information gain estimate noise | Improved |
Closest papers by meaning
- A Unified Framework for Entropy Search and Expected Improvement in Bayesian Optimization (ICML 2025 · similarity 0.91)
- Batched Energy-Entropy acquisition for Bayesian Optimization (NeurIPS 2024 · similarity 0.86)
- Provable Maximum Entropy Manifold Exploration via Diffusion Models (ICML 2025 · similarity 0.85)
- BONSAI: Bayesian Optimization with Natural Simplicity and Interpretability (NeurIPS 2026 · similarity 0.85)
- Optimizing Posterior Samples for Bayesian Optimization via Rootfinding (ICLR 2025 · similarity 0.85)
- ALINE: Joint Amortization for Bayesian Inference and Active Data Acquisition (NeurIPS 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
