Research map / Probabilistic and scientific modeling / Bayesian optimization
CoffeeBoost: Gradient Boosting Native Conformal Inference for Bayesian Optimization
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes CoffeeBoost, a Bayesian optimization framework whose surrogate is a stochastic gradient boosting ensemble with distribution-free conformal predictive systems and a GBDT-native boosted-residual difficulty estimator, evaluated on eight DBMS auto-tuning benchmarks.
Problem
Bayesian optimization for software configuration tuning relies on GP surrogates whose assumptions (continuity, Gaussianity, stationarity) fail on high-dimensional, heterogeneous, heteroscedastic system measurements, while existing GBDT uncertainty methods assume parametric predictive distributions o
Approach
Combine stochastic gradient boosting ensembles with conformal predictive systems, and use the boosted residuals across GBDT iterations as a native, training-free difficulty estimator for locally adaptive nonconformity scores, giving distribution-free predictive distributions for EI via Monte Carlo.
Key results
- CoffeeBoost improves final-tuned performance gain over SMAC by 41.1% on average across all eight benchmarks, and over GP-BOs by 109.7%.
- On the surrogate evaluation (YCSB), CoffeeBoost increases R2 by 10.1% and NCRPS by 13.1% relative to NGB-BOs.
- CoffeeBoost outperforms CoffeeBoost(ERC) by 6.6% and CoffeeBoost(NonLocal) by 8.7% in relative performance gain over SMAC.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use CoffeeBoost instead of SMAC | Database auto-tuning BO · Eight DBMS benchmarks | relative performance gain vs SMAC: +41.1% relative performance gain improvement on average | Helped |
| Use GBDT-native difficulty estimation instead of ERC (ablation) | CoffeeBoost · Eight DBMS benchmarks | relative performance gain vs SMAC: 6.6% | Helped |
| Use GBDT-native difficulty estimation instead of ERC (ablation) | CoffeeBoost · TWITTER and SYSBENCH | relative performance gain vs SMAC: 17.7% and 3.7% | Hurt |
| Use locally adaptive GBDT-native conformal inference instead of non-local EnbPI (ablation) | CoffeeBoost · Eight DBMS benchmarks | relative performance gain vs SMAC: 8.7% | Helped |
| Use locally adaptive GBDT-native conformal inference instead of non-local EnbPI (ablation) | CoffeeBoost · YCSB surrogate evaluation | NCRPS | Hurt |
| Select 20 equally spaced boosting iterations rather than all iterations (ablation) | CoffeeBoost difficulty estimator · YCSB surrogate evaluation | NCRPS: +0.05% | No clear effect |
| Select 20 equally spaced iterations from the later half of training (ablation) | CoffeeBoost difficulty estimator · YCSB surrogate evaluation | NCRPS: -0.18% | Hurt |
| Select the final 3 boosting iterations (ablation) | CoffeeBoost difficulty estimator · YCSB surrogate evaluation | NCRPS: -0.36% | Hurt |
Surprising findings
- No single pre-specified distributional assumption for NGBoost or PGBM consistently outperforms SMAC across benchmarks, even when the best family is chosen per benchmark.
- Among GBDT-based BOs, none outperforms SMAC on the SYSBENCH benchmark without conformalized ensembling.
Limitations
- Ensemble of GBDTs incurs higher computational training time and memory than a single GBDT.
- Predicting uncertainty for IBUG-style methods is too costly for BO, which the method avoids but at the cost of ensemble overhead.
- Evaluation is limited to database tuning benchmarks, with surrogate simulators replaced by transformer models.
Experimental setup
Datasets: database auto-tuning benchmarks (TWITTER, TATP, VOTER, SMALLBANK, YCSB, TPC-C, SYSBENCH, JOB)
Benchmarks: TWITTER TATP VOTER SMALLBANK YCSB TPC-C SYSBENCH JOB
Metrics: performance gain after tuning R2 NCRPS CRPS
Baselines: Matern-BO HEBO TuRBO HESBO SMAC NGB-BO PGBM-BO VEGB-BO SKOPT(GBQRT) Hyperboost CoffeeBoost(ERC) CoffeeBoost(NonLocal)
Builds on: stochastic gradient boosting ensemble conformal predictive systems EnbPI Error Re-weighted Conformal SMAC
Closest papers by meaning
- Adaptive Kernel Design for Bayesian Optimization Is a Piece of CAKE with LLMs (NeurIPS 2025 · similarity 0.87)
- Minimizing UCB: a Better Local Search Strategy in Local Bayesian Optimization (NeurIPS 2024 · similarity 0.87)
- Large Language Models to Enhance Bayesian Optimization (ICLR 2024 · similarity 0.86)
- A General Framework for User-Guided Bayesian Optimization (ICLR 2024 · similarity 0.86)
- BONSAI: Bayesian Optimization with Natural Simplicity and Interpretability (NeurIPS 2026 · similarity 0.86)
- Batched Energy-Entropy acquisition for Bayesian Optimization (NeurIPS 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.