Research map / Probabilistic and scientific modeling / Bayesian optimization
Constrained Bayesian Optimization under Partial Observations: Balanced Improvements and Provable Convergence
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes CBOB, a constrained Bayesian optimization method for partially observable constrained optimization problems that combines a balanced EI-times-DPOF acquisition function with heterogeneous-likelihood GPs fit by expectation propagation, with a convergence proof.
Problem
Expensive constrained black-box optimization where objective and constraint values are unobservable when a solution is infeasible (partially observable constrained optimization problems, POCOPs).
Approach
Replace the probability of feasibility in EIC with a dynamic POF (DPOF) that adds a boundary-based exploration function, giving balanced weights within known feasible regions and more weight to unknown ones; model partially observable constraints with heterogeneous-likelihood GPs (HLGPs) whose non-Gaussian likelihoods are handled by expectation propagation to produce virtual observations.
Key results
- On S1 problems CBOB achieved the best median best-evaluated value on KBF (-0.39), WBD (2.16) and Lunar (260), while on S2 it was best on Ackley (0.43), PVD (7198) and Swimmer (350).
- CBOB obtained a lower ratio of feasible evaluations than several baselines (e.g. 25.8% on XGB-H vs 51.2% for TSC), reflecting more exploration of infeasible regions.
- The paper reports that EICB outperforms EIC in all experiments after 100 function evaluations.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use CBOB instead of EIC | CBOB · KBF (S1) | BOV: -0.33 -> -0.39 | Helped |
| Use CBOB instead of MESC | CBOB · WBD (S1) | BOV: 2.28 -> 2.16 | Helped |
| Use CBOB instead of MESC | CBOB · XGB-H (S1) | BOV: 0.271 -> 0.274 | Hurt |
| Use CBOB instead of TSC | CBOB · Lunar (S1) | BOV: 247 -> 260 | Hurt |
| Use CBOB instead of MESC_h | CBOB · Ackley (S2) | BOV: 0.48 -> 0.43 | Helped |
| Use CBOB instead of MESC_h | CBOB · PVD (S2) | BOV: 7507 -> 7198 | Helped |
| Use CBOB instead of EIC_c | CBOB · MLP-D (S2) | BOV: 0.978 -> 0.982 | Hurt |
| Use CBOB instead of MESC_h | CBOB · Swimmer (S2) | BOV: 197 -> 350 | Hurt |
| Add DPOF-based balanced exploration to EIC (EICB) | EIC · All experiments after 100 FEs | best-evaluated objective | Helped |
| Replace GPC constraint models with HLGP (ablation) | EIC and MESC · S2 | BOV | Mixed |
| Use the more aggressive EMUB exploration function instead of POB (ablation) | EICB · KBF | best-evaluated objective | Hurt |
| Use the more aggressive EMUB exploration function instead of POB (ablation) | EICB · XGB-H | best-evaluated objective | Hurt |
Surprising findings
- Higher exploration reduces the feasible-evaluation ratio, yet final best values improve, suggesting infeasible evaluations still help optimization.
- EI-based CBO methods, including CBOB, show less efficiency on hyperparameter optimization problems.
Limitations
- Theoretical convergence is shown only under fully observable constraint values, not the partially observable setting used in experiments.
- CBOB's ratio of feasible evaluations is relatively low, so many evaluations are infeasible and provide little information.
- Exploration function uses a fixed beta=1.96 without tuning; the more aggressive MUB variant was only explored in a supplementary study.
Experimental setup
Datasets: California housing (XGBoost HPO) digits (MLP HPO) scikit-learn benchmarks OpenAI Gym Lunar Lander and Swimmer synthetic functions (Keane's bump, Ackley, welded beam, pressure vessel)
Benchmarks: Keane's bump function welded beam design XGBoost HPO (XGB-H) Lunar Landing Ackley with constraint pressure vessel design MLP HPO (MLP-D) Swimmer
Metrics: median best-evaluated value ratio of feasible evaluations
Baselines: EIC MESC TSC EIC with GPC MESC with GPC EIC with HLGP MESC with HLGP
Builds on: EIC (expected improvement with constraints) expectation propagation Gaussian process regression Trieste
Closest papers by meaning
- BayeSQP: Bayesian Optimization through Sequential Quadratic Programming (NeurIPS 2025 · similarity 0.87)
- Understanding High-Dimensional Bayesian Optimization (ICML 2025 · similarity 0.86)
- Bayesian Optimisation with Unknown Hyperparameters: Regret Bounds Logarithmically Closer to Optimal (NeurIPS 2024 · similarity 0.86)
- Relaxing the Additivity Constraints in Decentralized No-Regret High-Dimensional Bayesian Optimization (ICLR 2024 · similarity 0.86)
- Optimizing Posterior Samples for Bayesian Optimization via Rootfinding (ICLR 2025 · similarity 0.86)
- Stopping Bayesian Optimization with Probabilistic Regret Bounds (NeurIPS 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.