Research map / Probabilistic and scientific modeling / Bayesian optimization
Scalable Bayesian Optimization via Focalized Sparse Gaussian Processes
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes focalized sparse GP trained with a kernel-weighted variational loss emphasizing a search region, plus FocalBO, a hierarchical acquisition optimization that shrinks the search region around the current best, for high-dimensional BO with large offline and online data.
Problem
Bayesian optimization is limited by cubic GP cost and by overly smooth sparse GP surrogates when datasets are large and dimensions are high, especially with large offline datasets.
Approach
A focalized ELBO weights each training point's likelihood term by its maximum kernel value to the search region (with a regularizer on weight mass), so the sparse GP spends its inducing capacity on the region being optimized; FocalBO then hierarchically optimizes the acquisition over progressively halved, best-centered regions with an adaptive depth.
Key results
- On the 585-dimensional musculoskeletal pouring task with 2,000 offline points and 3,000 online budget, FocalBO outperforms baselines in maximum reward and convergence speed.
- On Design-Bench D'Kitty morphology design, FocalBO with TuRBO is reported as the first GP-based method to reach top-tier performance of prior MBO works, while other baselines struggle to improve.
- Focalized GP has smaller KL divergence to the exact GP over search regions than SVGP across search-region sizes (8000 training points, 50 trials).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use FocalBO instead of SVGP | Bayesian optimization with TS, EI, or PI · Synthetic functions (Shekel, Michalewicz, GP-sampled) | optimization performance | Helped |
| Use FocalBO instead of sparse-GP baselines | Robot morphology design · D’Kitty morphology design (Design-Bench) | optimization performance | Helped |
| Combine FocalBO with TuRBO instead of using other baselines | Robot morphology design · D’Kitty morphology design (Design-Bench) | optimization performance | Helped |
| Use FocalBO instead of other baselines | Musculoskeletal system control · Human arm pouring task | maximum reward | Helped |
| Use focalized GP instead of SVGP | TuRBO · High-dimensional problems | optimization performance | Helped |
| Use FocalBO instead of TuRBO | Bayesian optimization · Robot morphology design and human musculoskeletal system control | optimization performance | Helped |
| Train with focalized ELBO instead of the SVGP ELBO | Sparse GP prediction · Local search regions | KL divergence to exact GP posterior | Helped |
| Train with focalized GP instead of Exact GP or SVGP | GP surrogate · Ackley and Rastrigin search regions smaller than 0.5 | NLL and RMSE | Helped |
| Remove the focalized-loss regularization term (ablation) | Focalized GP · Local GP prediction | prediction performance | Hurt |
| Center the search region at the current best point rather than alternative centers (ablation) | FocalBO · GP-sampled 2D functions | distance from search-region center to global optimum | Helped |
| Use FocalBO for online-dominant optimization instead of sparse-GP baselines | Bayesian optimization · Ackley and Hartmann | optimization performance | Mixed |
Surprising findings
- Sparse GPs with few inducing points sample more diverse Thompson sampling batches than exact GPs, i.e. they are more explorative.
Limitations
- Theoretical regret guarantees are not derived; only intuition is given.
- Musculoskeletal model is not yet publicly released, available only on request.
- Optimization depth and search-region choices are heuristic and problem-dependent.
Experimental setup
Datasets: Design-Bench D'Kitty morphology dataset Shekel, Michalewicz, GP-sampled synthetic functions Ackley Hartmann Rastrigin musculoskeletal arm pouring task (SAC-generated muscle activation data)
Benchmarks: Shekel Michalewicz GP-sampled functions Ackley Hartmann Design-Bench D'Kitty musculoskeletal arm control
Metrics: best objective value over BO iterations NLL RMSE KL divergence to exact GP
Baselines: SVGP WOGP Vecchia GP TuRBO TuRBO with nearest neighbor GP
Builds on: stochastic variational Gaussian process TuRBO BoTorch
Closest papers by meaning
- Understanding High-Dimensional Bayesian Optimization (ICML 2025 · similarity 0.90)
- Informed Initialization for Bayesian Optimization and Active Learning (NeurIPS 2025 · similarity 0.88)
- Bayesian Optimization via Continual Variational Last Layer Training (ICLR 2025 · similarity 0.88)
- Standard Gaussian Process is All You Need for High-Dimensional Bayesian Optimization (ICLR 2025 · similarity 0.88)
- Approximation-Aware Bayesian Optimization (NeurIPS 2024 · similarity 0.87)
- Optimizing Posterior Samples for Bayesian Optimization via Rootfinding (ICLR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.