Research map / Probabilistic and scientific modeling / Bayesian optimization
BARK: A Fully Bayesian Tree Kernel for Black-box Optimization
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes BARK, a fully Bayesian tree kernel GP trained by MCMC over forest structures (BART-style priors, marginalized leaves) and used for Bayesian optimization via a MIP-optimized integrated UCB over mixed feature spaces.
Problem
Bayesian optimization over mixed-feature spaces needs well-calibrated surrogate uncertainty; existing tree-based surrogates lack principled uncertainty, and tree kernel GPs fit trees in a two-step, non-Bayesian way that overfits or ignores tree-structure uncertainty.
Approach
Treat the BART-style sum of trees as a GP with a tree-agreement kernel, marginalize leaf values analytically, and sample tree structures jointly with the GP via MCMC so each posterior sample is a GP; the acquisition is an integrated UCB solved globally with mixed-integer programming, with splits sampled uniformly over the domain instead of at data points.
Key results
- On UCI regression, BARK matches BART in NLPD and MSE, with a slight NLPD advantage attributed to the GP perspective.
- On the MAX material design benchmark, BART and BARK identify the best candidate in fewer iterations than GP-RBF.
- BARK's per-iteration cost is about 50 s to fit and 100 s to optimize the acquisition, the paper's main stated limitation; GP-RBF achieves the best final value on XGBoostMNIST, and BARK underperforms on DiscreteRosenbrock.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| sample splitting rules uniformly from the domain rather than at observed data points (ablation) | BARK · Bayesian optimization | exploration | Helped |
| sample splitting rules at observed data points rather than uniformly from the domain (ablation) | BARK · Bayesian optimization | optimization performance | Hurt |
| increase the number of posterior samples from the default 16 (ablation) | BARK · regression tasks | predictive performance | Helped |
| vary the node depth prior hyperparameters from the defaults (0.95, 2.0) (ablation) | BARK · regression tasks | NLPD | No clear effect |
| ablate acquisition-function optimization (ablation) | BARK and BART · MAX | simple regret | No clear effect |
| use BARK rather than LeafGP | Bayesian optimization · synthetic and applied benchmarks | optimization performance | Helped |
| use BARK rather than BARKPrior, adding posterior inference over tree kernels | BARK · applied benchmarks | optimization performance | Helped |
| use BARK rather than GP-RBF | Bayesian optimization · XGBoostMNIST and other applied benchmarks | final objective value | Mixed |
| use BARK rather than GP-RBF | Bayesian optimization · DiscreteRosenbrock | optimization performance | Hurt |
Surprising findings
- BART with grid-based UCB evaluation does not outperform LeafGP and BARK even on TreeFunction benchmarks drawn from the BART prior, indicating the acquisition optimization matters more than the surrogate alone.
- Sampling splits uniformly over the domain rather than at data points (the BART default) is needed for good exploration in BO.
Limitations
- Wall-time is substantially higher than competing methods (about 50 s fit and 100 s acquisition optimization), so it suits only expensive black-box functions.
- Piecewise-constant, discontinuous functions violate standard assumptions for regret bounds; no regret bound is proven.
- Underperforms smooth-kernel GP-RBF on some benchmarks such as DiscreteRosenbrock and continuous functions.
Experimental setup
Datasets: UCI regression datasets TreeFunction TreeFunctionCat DiscreteAckley DiscreteRosenbrock SVRBench XGBoostMNIST CCOBench PestControl MAX phases dataset Hartmann Styblinski-Tang
Benchmarks: UCI regression TreeFunction TreeFunctionCat DiscreteAckley DiscreteRosenbrock SVRBench XGBoostMNIST CCOBench PestControl MAX Hartmann Styblinski-Tang
Metrics: NLPD MSE simple regret minimum observed value wall-clock time
Baselines: BART GP-RBF LeafGP SMAC Entmoot BARKPrior
Builds on: BART forest kernel LeafGP Entmoot
Closest papers by meaning
- Automated Kernel Discovery Towards Understanding High-dimensional Bayesian Optimization (NeurIPS 2026 · similarity 0.88)
- Adaptive Kernel Design for Bayesian Optimization Is a Piece of CAKE with LLMs (NeurIPS 2025 · similarity 0.87)
- BayeSQP: Bayesian Optimization through Sequential Quadratic Programming (NeurIPS 2025 · similarity 0.87)
- Kernel-based guarantees for nonlinear parametric models in Bayesian optimization (NeurIPS 2026 · similarity 0.87)
- BONSAI: Bayesian Optimization with Natural Simplicity and Interpretability (NeurIPS 2026 · similarity 0.87)
- Standard Gaussian Process is All You Need for High-Dimensional Bayesian Optimization (ICLR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.