Research map / Probabilistic and scientific modeling / Bayesian optimization
Regional Expected Improvement for Efficient Trust Region Selection in High-Dimensional Bayesian Optimization
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes regional expected improvement (REI, and qREI), an acquisition function that averages EI over a trust region to choose promising trust-region centers for TuRBO in medium-to-high-dimensional Bayesian optimization.
Problem
Bayesian optimization in medium-to-high-dimensional (D>10) spaces suffers from limited evaluations; trust-region methods like TuRBO focus on local search and can stagnate in local optima, with restarts placed by conventional acquisition functions.
Approach
Scalarize the GP acquisition function over a trust region by averaging EI (or qEI/TS) within the region, so trust-region centers are chosen by regional rather than local acquisition values; a theorem shows the region-averaging operator does not increase the RKHS-based regret bound.
Key results
- TuRBO-1-LogEI-qREI improves over TuRBO-1-LogEI on all tested medium-dimensional problems, with significant differences on HPA101-0 and HPA102-1.
- On the 60-dimensional rover problem, TuRBO-1-LogEI-qREI variants achieved the best results after more than 1000 evaluations.
- Theorem shows the region-averaged objective S f satisfies U_N(Sf) <= U_N(f), suggesting lower regret-bound complexity for region-averaged problems.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add qREI-based trust-region selection at initialization and restarts | TuRBO-1-LogEI · HPA101-0 | objective value | Helped |
| Add qREI-based trust-region selection at initialization and restarts | TuRBO-1-LogEI · HPA102-1 | objective value | Helped |
| Add qREI-based trust-region selection at initialization and restarts | TuRBO-1-LogEI · HPA103-1 | objective value | Helped |
| Add qREI-based trust-region selection at initialization and restarts | TuRBO-1-LogEI · Rover trajectory planning | objective value | Helped |
| Add qREI-based trust-region selection at initialization and restarts | TuRBO-1-LogEI · HPA101-2 | objective value | Helped |
| Add qREI-based trust-region selection at initialization and restarts | TuRBO-1-LogEI · MOPTA08 | objective value | No clear effect |
| Use qREI only during restarts, retaining standard initialization (ablation) | TuRBO-1-LogEI-qREI(restart) · HPA101-2 and MOPTA08 | objective value | No clear effect |
| Use qREI for initial trust-region selection, reducing points available for local search (ablation) | TuRBO-1-LogEI-qREI · HPA102-1 and high-dimensional problems, up to 300 evaluations | objective value | Hurt |
| Select trust regions using region-averaged LogEI and qREI rather than pointwise LogEI (ablation) | TuRBO-1-LogEI · Medium-dimensional problems | final objective value | Helped |
| Select trust regions using region-averaged UCB rather than pointwise UCB (ablation) | TuRBO-1-LogEI · Medium-dimensional problems | objective value | Helped |
| Add qREI-based trust-region selection (ablation) | TuRBO-5-LogEI/TS · Medium-dimensional problems | objective value | Helped |
| Add qREI-based trust-region selection (ablation) | TuRBO-1-TS · Evaluated benchmarks | objective value | Helped |
Surprising findings
- On MOPTA08 the qREI effect is very limited because all methods converge to solutions of similar quality regardless of trust region.
- Using qREI for initial trust region selection reduces the samples available for local search, decreasing performance in low-budget settings (up to 300 evaluations).
Limitations
- Restarts become rare in high dimensions, so the qREI effect is limited to initial trust region selection for many problems.
- Benefit is small on problems like MOPTA08 where results are insensitive to trust region choice.
- Low-budget performance can degrade because qREI initialization consumes samples used for local search.
Experimental setup
Datasets: Human-powered aircraft (HPA) problems Rover trajectory planning MOPTA08
Benchmarks: HPA101-0 HPA102-1 HPA103-1 HPA101-2 Rover trajectory planning MOPTA08
Metrics: best objective value vs. evaluations final objective value distribution Wilcoxon signed-rank and rank-sum tests
Baselines: TuRBO-1-LogEI DSP Bounce SAASBO GP-LogEI CMA-ES
Builds on: TuRBO LogEI BAxUS SAASBO Monte Carlo qEI
Closest papers by meaning
- Understanding High-Dimensional Bayesian Optimization (ICML 2025 · similarity 0.87)
- Bayesian Optimization with Fisher Information Geometry: Gradient Bounds and Trust-Region Methods (NeurIPS 2026 · similarity 0.86)
- Principled Bayesian Optimization in Collaboration with Human Experts (NeurIPS 2024 · similarity 0.86)
- Trust Region Constrained Measure Transport in Path Space for Stochastic Optimal Control and Inference (NeurIPS 2025 · similarity 0.86)
- Trust Region Methods for Nonconvex Stochastic Optimization beyond Lipschitz Smoothness (AAAI 2024 · similarity 0.86)
- Vanilla Bayesian Optimization Performs Great in High Dimensions (ICML 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.