Research map / Probabilistic and scientific modeling / Bayesian optimization
High-dimensional Bayesian optimization
Scale Bayesian optimization to high-dimensional and discrete spaces using structured kernels and search-space learning.
34 accepted papers at ICML, NeurIPS, ICLR, CVPR and AAAI (2017–2026) · an approach in Bayesian optimization, keyword cluster “black box · box optimization · offline”
State History Tried Checklist Preprints Who Export
Where it stands
Not enough benchmark data Fewer than four papers report the same benchmark, so there is no record to follow.
9 papers in 2023–24 and 6 in 2025–26 (at ICML, NeurIPS, ICLR, CVPR and AAAI); its share of the topic shrank, from 9.8% to 4.5%.
How this is decided
A benchmark is tracked when at least four papers of this approach report their own method on the same dataset and metric in at least two different years. Its record is the best number reported up to 2024 compared with the best reported in 2025–26: still improving means at least 1% better, flat means at most 0.2% better with at least two papers from 2025–26 trying. Numbers on a different scale from the rest (a fraction among percentages) and far outliers are left out. Results come from different papers with different setups, so they are signals, not head-to-head comparisons. The share of papers is counted at ICML, NeurIPS, ICLR, CVPR and AAAI.
How it got here
Most cited paper of each year
- 2017 · Multi-fidelity Bayesian Optimisation with Continuous Approximations (ICML 2017 · 104 citations)
- 2018 · Scalable Hyperparameter Transfer Learning (NeurIPS 2018 · 77 citations)
- 2019 · Learning search spaces for Bayesian optimization: Another view of hyperparameter transfer learning (NeurIPS 2019 · 36 citations)
- 2020 · Learning Search Space Partition for Black-box Optimization using Monte Carlo Tree Search (NeurIPS 2020 · 30 citations)
- 2021 · Think Global and Act Local: Bayesian Optimisation over High-Dimensional Categorical and Mixed Search Spaces (ICML 2021 · 14 citations)
- 2022 · Fourier Representations for Black-Box Optimization over Categorical Variables (AAAI 2022 · 5 citations)
- 2023 · CO-BED: Information-Theoretic Contextual Optimization via Bayesian Experimental Design (ICML 2023 · 1 citation)
- 2024 · Grey-Box Bayesian Optimization for Sensor Placement in Assisted Living Environments (AAAI 2024 · 3 citations)
- 2025 · BayeSQP: Bayesian Optimization through Sequential Quadratic Programming (NeurIPS 2025 · 1 citation)
- 2026 · Few-Shot Design Optimization by Exploiting Auxiliary Information (ICML 2026 · 0 citations)
Earliest papers here
- Multi-fidelity Bayesian Optimisation with Continuous Approximations (ICML 2017 · 104 citations)
- Batched High-dimensional Bayesian Optimization via Structural Kernel Learning (ICML 2017 · 20 citations)
- Scalable Hyperparameter Transfer Learning (NeurIPS 2018 · 77 citations)
What has been tried
32 papers here report 291 measured changes (a component added, removed or swapped, compared with a baseline): helped 67%, no clear effect 8%, hurt 18%, mixed 8%. Papers mostly publish changes that helped, so "helped" is over-represented.
Ablations in recent papers
| What was changed | Measured on | Result | Effect | Paper |
|---|---|---|---|---|
| Reduce the number of sub-samples K below d | constrained within-model · performance | Helped | BayeSQP: Bayesian Optimization through Sequential Quadratic Programming (NeurIPS 2025 · 1 citation) | |
| Add EBM global exploration signals | Branin 2D · final objective | 8.95 \(±\) 0.20 -> 9.30 \(±\) 0.12 | Helped | Optimizing the Unknown: Black Box Bayesian Optimization with Energy-Based Model and Reinforcement Learning (NeurIPS 2025 · 1 citation) |
| set removal-budget hyperparameter α to 1/4 | 11 benchmarks (10 synthetic functions and one real-world experiment) · normalized average regret | Helped | This Too Shall Pass: Removing Stale Observations in Dynamic Bayesian Optimization (NeurIPS 2024 · 1 citation) | |
| Train the GP on the full target dataset rather than only selected points in the chosen subspace | LunarLander, RobotPush, and Rover, mixed transfer · search performance | Helped | Monte Carlo Tree Search based Space Transfer for Black Box Optimization (NeurIPS 2024 · 0 citations) | |
| sample splitting rules uniformly from the domain rather than at observed data points | Bayesian optimization · exploration | Helped | BARK: A Fully Bayesian Tree Kernel for Black-box Optimization (ICML 2025) | |
| add reweighted training focused on high-scoring data | Rastrigin-200D and HalfCheetah-102D · optimization performance | Helped | Posterior Inference with Diffusion Models for High-dimensional Black-box Optimization (ICML 2025) |
What reviewers will expect
Datasets and baselines named in the 5 papers of this approach from 2025–2026 that have a full-text summary. Leaving out the common ones is a frequent review question.
Datasets
- Ackley · 2 of 5 papers
- Rosenbrock · 2 of 5 papers
Baselines
- TuRBO · 3 of 5 papers
Recent preprints
Loading recent arXiv preprints…
Who publishes on it
Institutions on the most papers of this approach since 2023 (of 15 papers; from OpenAlex and the papers' affiliations).
- Amazon · 2 papers
- Microsoft · 2 papers
- University of Oxford · 2 papers
Use it
Download BibTeX of 28 key papersOpen the topic in the interactive map
Get alerts. Describe your idea in scime atlas and save it (free scime account): you get a weekly email when new preprints land close to it.
Related approaches
Same keyword cluster
- Offline black-box optimization with surrogates (56 papers)
Elsewhere in the topic
- Acquisition function design (97 papers)
- Local and high-dimensional Bayesian optimization (90 papers)
- Multi-fidelity hyperparameter optimization (44 papers)
- Regret bounds for Bayesian optimization (43 papers)
- Multi-objective Bayesian optimization (33 papers)
Benchmark numbers, ablations and baseline numbers are extracted by AI from each paper's full text and can be wrong; the paper is the authority. Data: results index of 2026-11-02, citation counts from OpenAlex (2026-10-09).