Research map / Probabilistic and scientific modeling / Bayesian optimization
PABBO: Preferential Amortized Black-Box Optimization
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Fully amortizes preferential Bayesian optimization by meta-training a transformer neural process with RL and an auxiliary BCE preference-prediction loss, giving much faster inference than GP-based PBO.
Problem
Preferential Bayesian optimization needs costly approximate GP inference and acquisition optimization at every step, which is too slow for interactive human-in-the-loop use from pairwise duel feedback.
Approach
Meta-learn a transformer neural process that directly outputs acquisition values for candidate query pairs conditioned on preference history, trained with RL (discounted cumulative simple regret reward) plus an auxiliary binary-classification preference-prediction loss to handle sparse rewards.
Key results
- Average 12-fold inference speed-up over the fastest GP-based strategy across 5 problems, and orders-of-magnitude faster overall per the abstract.
- PABBO consistently ranks best or second best in simple regret on in-distribution synthetic GP draws and achieves lowest simple regret on out-of-distribution Forrester, Branin, and Beale.
- On HPO-B, PABBO achieves the best performance, closely followed by qNEI; on Candy and Sushi it is competitive, placing second and third respectively.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use PABBO instead of the fastest GP-based strategy | Preferential black-box optimization · 5 synthetic problems | inference speed: 12-fold | Helped |
| Replace latent-value rewards with rankings during pre-training (ablation) | PABBO · Forrester and other tested functions | simple regret | No clear effect |
| Use a smaller discount factor (ablation) | PABBO · Forrester | simple regret | Mixed |
| Increase the discount factor (ablation) | PABBO · Forrester | cumulative simple regret | Helped |
| Increase query set size from 256 to 1024 (ablation) | PABBO · 2D Branin | simple regret | Helped |
| Pre-train on a mixture of kernels instead of only an RBF kernel (ablation) | PABBO · 30 2D GP draws | simple regret | Helped |
| Add a prediction-task-only warm-up phase (ablation) | PABBO · Three 2D functions | optimization performance | Helped |
| Vary the auxiliary loss weight after including warm-up (ablation) | PABBO · Three 2D functions | optimization performance | No clear effect |
| Include t/T as input to the acquisition head (ablation) | PABBO · Forrester | simple regret | Helped |
| Increase the query budget while providing t/T to the acquisition head (ablation) | PABBO · Forrester | cumulative simple regret | Helped |
| Use a larger discount factor (ablation) | PABBO · Forrester | simple regret | Mixed |
Surprising findings
- Random acquisition often outperforms certain GP-based PBO baselines on synthetic functions.
- Replacing latent function values with pure rankings during pre-training did not significantly degrade performance.
Limitations
- Query set size scales with input dimension, increasing inference time in high dimensions.
- Model handles only fixed input dimensionality, requiring separate models per dimension.
- Performance drops on out-of-distribution multimodal functions such as Hartmann, and it does not model the human user's comparison noise.
Experimental setup
Datasets: synthetic GP draws HPO-B Candy power ranking dataset Sushi preference dataset
Benchmarks: Forrester Branin Beale HPO-B Candy Sushi
Metrics: simple regret cumulative simple regret inference time
Baselines: qEUBO qEI qNEI qTS MPES Random
Builds on: Deep Adaptive Design Transformer neural processes Conditional neural processes BoTorch
Closest papers by meaning
- Principled Preferential Bayesian Optimization (ICML 2024 · similarity 0.88)
- DT-PBO: an Interpretable Tree-based Surrogate Model for Preferential Bayesian Optimization (NeurIPS 2026 · similarity 0.87)
- Joint Composite Latent Space Bayesian Optimization (ICML 2024 · similarity 0.86)
- Posterior Inference with Diffusion Models for High-dimensional Black-box Optimization (ICML 2025 · similarity 0.86)
- A General Framework for User-Guided Bayesian Optimization (ICLR 2024 · similarity 0.86)
- Bayesian Optimization from Human Feedback: Near-Optimal Regret Bounds (ICML 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.