Research map / Probabilistic and scientific modeling / Bayesian optimization
BOIDS: High-Dimensional Bayesian Optimization via Incumbent-Guided Direction Lines and Subspace Embeddings
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes BOIDS, a high-dimensional Bayesian optimization method that runs line-based acquisition optimization along PSO-style incumbent-guided direction lines, selects lines via Thompson sampling bandit, and adds BAxUS subspace embedding.
Problem
Scaling Bayesian optimization to high-dimensional expensive black-box problems (100-500 dimensions), where acquisition optimization over the full space and over-exploration degrade performance.
Approach
Replace LineBO's uniformly random guiding lines with PSO-inspired directions built from personal and global incumbents, select the best line via Thompson-sampling bandit over GP posterior samples, optimize the acquisition function over the full space with distance-to-incumbent objectives via NSGA-II, and embed in expanding BAxUS subspaces.
Key results
- BOIDS significantly outperforms all baselines on the synthetic Ackley-100D, Branin-500D and Hartmann-500D problems according to the paper's comparison figure.
- On LassoDNA-180D, BOIDS converges much faster than RDUCB and CMA-BO, which reach similarly optimal solutions.
- On HalfCheetah-102D, ALEBO (d_A=10) is competitive with BOIDS but needs careful tuning and is prohibitively expensive (up to 40 hours for 500 iterations).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add incumbent-guided direction lines instead of uniformly random guiding lines (ablation) | BOIDS · LassoDNA-180D | optimization performance | Helped |
| add incumbent-guided direction lines instead of uniformly random guiding lines (ablation) | BOIDS · HalfCheetah-102D | optimization performance | Helped |
| use MAB-based adaptive line selection instead of random line selection (ablation) | BOIDS · LassoDNA-180D | optimization performance | No clear effect |
| use MAB-based adaptive line selection instead of random line selection (ablation) | BOIDS · HalfCheetah-102D | optimization performance | No clear effect |
| use incumbent-guided line-based optimization instead of optimizing the acquisition function over the entire input space (ablation) | BOIDS · LassoDNA-180D | optimization performance | Helped |
| use incumbent-guided line-based optimization instead of optimizing the acquisition function over the entire input space (ablation) | BOIDS · HalfCheetah-102D | optimization performance | Helped |
| use subspace embedding rather than optimize directly in the input dimension (ablation) | BOIDS · LassoDNA-180D | optimization performance | No clear effect |
| use subspace embedding rather than optimize directly in the input dimension (ablation) | BOIDS · HalfCheetah-102D | optimization performance | Helped |
Surprising findings
- The MAB-based line selection is the least important component in the ablation, as removing it does not significantly degrade performance.
- BOIDS starts slower than TuRBO on Mopta-124D due to its initial focus on low-dimensional subspaces.
Limitations
- Higher per-iteration runtime than LineBO, BO and BAxUS (around 12-23 s per iteration).
- Simple regret bound relies on RKHS, 1D kernel information-gain and alpha-convexity/beta-smoothness assumptions; the global guarantee depends on the embedding containing the optimum with probability below one for d_A < d.
- Several baselines run with fewer iterations (SAASBO 100, ALEBO 500, RDUCB time-limited), so comparisons are not on equal budgets.
Experimental setup
Datasets: Ackley-100D Branin-500D Hartmann-500D Mopta-124D LassoDNA-180D HalfCheetah-102D
Benchmarks: Ackley-100D Branin-500D Hartmann-500D Mopta-124D LassoDNA-180D HalfCheetah-102D
Metrics: best function value found vs iterations average runtime per iteration
Baselines: Standard BO LineBO BAxUS HESBO ALEBO SAASBO TuRBO CMA-BO RDUCB CMA-ES PSO
Builds on: LineBO BAxUS Particle Swarm Optimization Thompson sampling NSGA-II
Closest papers by meaning
- Vanilla Bayesian Optimization Performs Great in High Dimensions (ICML 2024 · similarity 0.90)
- Posterior Inference with Diffusion Models for High-dimensional Black-box Optimization (ICML 2025 · similarity 0.89)
- Understanding High-Dimensional Bayesian Optimization (ICML 2025 · similarity 0.89)
- Joint Composite Latent Space Bayesian Optimization (ICML 2024 · similarity 0.88)
- Standard Gaussian Process is All You Need for High-Dimensional Bayesian Optimization (ICLR 2025 · similarity 0.88)
- Minimizing UCB: a Better Local Search Strategy in Local Bayesian Optimization (NeurIPS 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.