Research map / Probabilistic and scientific modeling / Bayesian optimization
Explaining Hyperparameter Optimization via Partial Dependence Plots
Julia Moosbauer, Julia Herbinger, Giuseppe Casalicchio, Marius Lindauer, Bernd Bischl
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Partition PDPs into 3 uncertainty-based sub-regions instead of using only a global PDP (ablation) | GP surrogate PDPs on Styblinski-Tang · Styblinski-Tang, d=3, 5, 8 | relative MC reduction (%): 13.64; 36.92; 34.84; 15.45; 37.28; 33.06; 9.67; 23.03; 19.84 | Improved |
| Partition PDPs into 3 uncertainty-based sub-regions instead of using only a global PDP (ablation) | GP surrogate PDPs on Styblinski-Tang · Styblinski-Tang, d=3, 5, 8 | relative NLL improvement (%): 10.92; 7.70; -1.62; 6.05; 7.80; -1.93; 2.40; 3.30; 4.29 | Mixed |
| Increase uncertainty-based splits from 1 to 3 (ablation) | GP surrogate PDPs on Styblinski-Tang · Styblinski-Tang, d=3, 5, 8 | relative MC reduction (%): 7.65 -> 13.64; 12.86 -> 36.92; 16.52 -> 34.84; 6.63 -> 15.45; 19.67 -> 37.28; 11.99 -> 33.06; 3.58 -> 9.67; 8.86 -> 23.0 | Improved |
| Increase uncertainty-based splits from 1 to 3 (ablation) | GP surrogate PDPs on Styblinski-Tang · Styblinski-Tang, d=3, 5, 8 | relative NLL improvement (%): 5.89 -> 10.92; 4.78 -> 7.70; 2.77 -> -1.62; 2.82 -> 6.05; 4.05 -> 7.80; -3.86 -> -1.93; 0.84 -> 2.40; 1.51 -> 3.30; 1.53 | Mixed |
| Partition the PDP into 6 uncertainty-based regions rather than using the global PDP (ablation) | GP surrogate PDPs on LCBench · LCBench, 35 classification tasks | relative MC reduction (%): 40.8 (14.9); 50.2 (13.7); 49.7 (15.4); 51.1 (15.2); 51.7 (14.5); 30.6 (16.4); 36.3 (22.6) | Improved |
| Partition the PDP into 6 uncertainty-based regions rather than using the global PDP (ablation) | GP surrogate PDPs on LCBench · LCBench, 35 classification tasks | relative OC reduction (%): 61.9 (13.5); 57.6 (14.4); 62.4 (11.9); 58.6 (12.7); 58.3 (12.7); 50.9 (16.6); 61.0 (13.1) | Improved |
| Partition the PDP into 6 uncertainty-based regions rather than using the global PDP (ablation) | GP surrogate PDPs on LCBench · LCBench, 35 classification tasks | relative NLL improvement (%): 19.8 (19.5); 17.9 (20.5); 17.4 (18.2); 24.6 (22.0); 19.7 (21.7); 13.8 (32.5); 11.9 (19.7) | Improved |
| Use a diagonal-only variance approximation instead of modeled pairwise covariance (ablation) | Probabilistic surrogate PDP uncertainty estimates · Styblinski-Tang PDP kernel-misspecification experiment | NLL variability: 5.10 (5.91) -> 4.62 (0.32); 4.33 (3.72) -> 4.89 (0.28); 4.24 (2.12) -> 5.13 (0.17) | Improved |
Closest papers by meaning
- Interactive Hyperparameter Optimization in Multi-Objective Problems via Preference Learning (AAAI 2024 · similarity 0.88)
- Meta-Surrogate Benchmarking for Hyperparameter Optimization (NeurIPS 2019 · similarity 0.88)
- HPO-B: A Large-Scale Reproducible Benchmark for Black-Box HPO based on OpenML (NeurIPS 2021 · similarity 0.88)
- Few-Shot Bayesian Optimization with Deep Kernel Surrogates (ICLR 2021 · similarity 0.87)
- UQ-Guided Hyperparameter Optimization for Iterative Learners (NeurIPS 2024 · similarity 0.86)
- Efficient Hyperparameter Optimization with Adaptive Fidelity Identification (CVPR 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
