Research map / Probabilistic and scientific modeling / Bayesian optimization
FunBO: Discovering Acquisition Functions for Bayesian Optimization with FunSearch
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Extends FunSearch to evolve Bayesian optimization acquisition functions written as code, scored by full BO loops on a set of training objectives, yielding interpretable AFs that generalize in and out of distribution.
Problem
Hand-designed acquisition functions in Bayesian optimization perform inconsistently across problems, while learned AFs (meta-learning, RL, transformers) often fail to generalize outside their training function class.
Approach
Treat acquisition function design as algorithm discovery: an LLM iteratively mutates AF code inside a FunSearch-style evolutionary database, with fitness computed by running a full BO loop on training objectives, and selecting on validation objectives. Output is interpretable code rather than a neural network.
Key results
- FunBO's discovered AF outperforms general-purpose AFs (EI, UCB, PI, mean) on nine out-of-distribution test functions in the OOD-Bench experiment.
- On the HPO AdaBoost task the FunBO-found AF outperforms all other AFs, while on SVM it performs similarly to general-purpose or MetaBO AFs.
- In the few-shot Ackley experiment, the FunBO adapted AF outperforms both general-purpose AFs and FSAF.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use the FunBO-discovered acquisition function instead of general-purpose acquisition functions | Bayesian optimization · OOD-Bench global optimization functions | normalized average simple regret | Helped |
| Use the OOD-Bench FunBO acquisition function instead of function-specific acquisition functions | Bayesian optimization · OOD-Bench global optimization functions | normalized average simple regret | Mixed |
| Use a function-class-specific FunBO acquisition function instead of general-purpose or MetaBO acquisition functions | Bayesian optimization · ID-Bench: Branin, Goldstein-Price, and Hartmann-3 | normalized average simple regret | Mixed |
| Use a function-class-specific FunBO acquisition function instead of other acquisition functions | Bayesian optimization · HPO-ID AdaBoost | normalized average simple regret | Helped |
| Use a function-class-specific FunBO acquisition function instead of other acquisition functions | Bayesian optimization · HPO-ID SVM | normalized average simple regret | No clear effect |
| Use the FunBO acquisition function instead of other acquisition functions | Bayesian optimization · GPs-ID, d=4 | normalized average simple regret | Helped |
| Use the FunBO acquisition function instead of MetaBO | Bayesian optimization · GPs-ID, d=3 | normalized average simple regret | Helped |
| Adapt the initial FunBO acquisition function using 5 Ackley instances | Bayesian optimization · Few-shot Ackley | normalized average simple regret | Helped |
Surprising findings
- The dimensionality-agnostic MetaBO baseline is not the strongest comparator; the discovered AF is described as slightly worse than MetaBO and customized FunBO AFs on HPO tasks.
- Variance in the quality of discovered AFs is high, and finding AFs that generalize across function classes required multiple FunBO runs.
Limitations
- Evaluating each candidate AF requires running a full BO loop per training function, which is expensive and limits scalability to larger training sets or multi-objective settings.
- High variance in discovered AF quality across runs, requiring multiple runs to find well-generalizing AFs.
- Only the simplest single-output BO with GP surrogate and closed-form AFs is studied.
Experimental setup
Datasets: Ackley Levy Schwefel Rosenbrock Branin Goldstein-Price Hartmann HyLAP SVM and AdaBoost loss datasets GP prior samples (RBF kernel)
Benchmarks: Sphere Styblinski-Tang Weierstrass Beale Branin Michalewicz Goldstein-Price Hartmann-3 Hartmann-6 HPO SVM and AdaBoost (HyLAP) GP-sampled functions d=3 and d=4 Ackley few-shot
Metrics: normalized average simple regret
Baselines: Expected Improvement Upper Confidence Bound Probability of Improvement posterior mean MetaBO FSAF random selection
Builds on: FunSearch MetaBO FSAF island model
Closest papers by meaning
- Batched Energy-Entropy acquisition for Bayesian Optimization (NeurIPS 2024 · similarity 0.89)
- Large Language Models to Enhance Bayesian Optimization (ICLR 2024 · similarity 0.89)
- A General Framework for User-Guided Bayesian Optimization (ICLR 2024 · similarity 0.89)
- A Unified Framework for Entropy Search and Expected Improvement in Bayesian Optimization (ICML 2025 · similarity 0.89)
- BONSAI: Bayesian Optimization with Natural Simplicity and Interpretability (NeurIPS 2026 · similarity 0.88)
- Simple Extensions of Single-Objective Acquisition Functions and Hedge Strategies for Multi-Objective Bayesian Optimization (NeurIPS 2026 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.