Research map / Probabilistic and scientific modeling / Bayesian optimization
Boundary Exploration for Bayesian Optimization With Unknown Physical Constraints
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes BE-CBO, a Bayesian optimization method for unknown binary physical constraints that models feasibility with deep ensembles and explores the feasible/infeasible boundary using a dynamic uncertainty-based bound on EI maximization.
Problem
Bayesian optimization of expensive black-box objectives under unknown binary (pass/fail) physical constraints, where infeasible designs yield no objective value and optima tend to lie on the feasibility boundary.
Approach
Model the binary feasibility constraint with a deep ensemble of MLP classifiers (trained with variational ELBO on a latent Gaussian output) rather than a GP classifier, and restrict EI maximization to a band around the 0.5 feasibility boundary whose width is set by the ensemble's uncertainty.
Key results
- BE-CBO shows top performance and low variance across most of 12 benchmarks over 200 evaluations (10 seeds), though slightly outperformed by SCBO on the 30D cantilever beam.
- Ablation: replacing deep ensembles with GP classifiers causes significant performance drop on about half of the benchmark problems.
- All nine real-world problems have their global optimum on the constraint boundary (at least one active constraint).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use BE-CBO with deep ensembles and boundary exploration instead of the compared baselines | Bayesian optimization with unknown binary physical constraints · Synthetic and real-world benchmarks | best objective value | Helped |
| Use BE-CBO instead of SCBO | Bayesian optimization with unknown binary physical constraints · 30D cantilever beam design | best objective value | Hurt |
| Replace deep-ensemble constraint classification with a GP classifier (ablation) | BE-CBO · Half of the benchmark problems | best objective value | Hurt |
| Use deep ensembles instead of GP classifiers for constraint classification (ablation) | BE-CBO · All benchmark problems | balanced accuracy | Helped |
| Train deep ensembles with maximum likelihood estimation instead of variational inference (ablation) | BE-CBO constraint classifier · Benchmarks | optimization performance | Hurt |
| Train deep ensembles with variational inference instead of maximum likelihood estimation (ablation) | BE-CBO constraint classifier · Benchmarks | optimization performance | Helped |
| Vary ensemble size, hidden layers, neurons per layer, and learning rate within the tested ranges (ablation) | BE-CBO deep ensembles · Benchmarks | overall performance | No clear effect |
| Replace EI with UCB as the acquisition function (ablation) | BE-CBO · Benchmark problems | best objective value | No clear effect |
| Replace boundary exploration with CEI-like constraint treatment (ablation) | BE-CBO · Benchmark problems | optimization performance | Hurt |
| Replace boundary exploration with SCBO-like constraint treatment (ablation) | BE-CBO · Benchmark problems | optimization performance | Hurt |
| Use boundary exploration with deep ensembles on synthetic problems whose optimum is shifted into the feasible interior | BE-CBO · Modified Townsend, Simionescu, and LSQ | best objective value | Helped |
| Use boundary exploration rather than searching only within the feasible region | BE-CBO · LSQ | number of evaluations to discover the global optimum | Helped |
Surprising findings
- Training deep ensembles with maximum likelihood (BCE) gave poorly calibrated uncertainty and degraded BO performance; variational ELBO training was needed.
- Using deep ensembles for regression of the objective did not improve performance, according to the authors' prior observation.
Limitations
- Only tested on continuous design spaces and single-objective problems.
- Does not account for costs of evaluating infeasible samples.
- No theoretical global convergence analysis under unknown constraints.
Experimental setup
Datasets: Townsend function Simionescu function LSQ function three-bar truss design tension-compression string design welded beam design gas transmission compressor design pressure vessel design speed reducer design planetary gear train design rolling element bearing design cantilever beam design
Benchmarks: Townsend Simionescu LSQ three-bar truss tension-compression string welded beam gas transmission compressor pressure vessel speed reducer planetary gear train rolling element bearing cantilever beam
Metrics: current best objective value vs. evaluations feasibility ratio balanced accuracy runtime
Baselines: CEI SCBO SVM-CBO random search GP classifier constraint modeling
Builds on: Expected Improvement CEI SCBO SVM-CBO Deep Ensembles
Closest papers by meaning
- Bayesian Optimization over Discrete and Mixed Spaces via Probabilistic Reparameterization (NeurIPS 2022 · similarity 0.89)
- Optimizing the Unknown: Black Box Bayesian Optimization with Energy-Based Model and Reinforcement Learning (NeurIPS 2025 · similarity 0.87)
- Informed Initialization for Bayesian Optimization and Active Learning (NeurIPS 2025 · similarity 0.87)
- BONSAI: Bayesian Optimization with Natural Simplicity and Interpretability (NeurIPS 2026 · similarity 0.87)
- Minimizing UCB: a Better Local Search Strategy in Local Bayesian Optimization (NeurIPS 2024 · similarity 0.86)
- Understanding High-Dimensional Bayesian Optimization (ICML 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.