Research map / Probabilistic and scientific modeling / Bayesian optimization
BayeSQP: Bayesian Optimization through Sequential Quadratic Programming
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes BayeSQP, a local black-box optimizer that uses joint GP surrogates of function, gradient and Hessian from zeroth-order data to solve an uncertainty-aware second-order cone SQP subproblem, followed by constrained Thompson sampling line search.
Problem
Constrained black-box optimization in high dimensions where only zeroth-order (function value) feedback is available, so gradients and Hessians must be inferred from noisy observations.
Approach
Merges SQP structure with BO by modeling objective and constraints with second-order GPs (function, gradient, Hessian mean) and formulating the SQP subproblem with value-at-risk and probabilistic constraint satisfaction, which reduces to a second-order cone program; the step size is chosen by constrained Thompson sampling on the line segment.
Key results
- x
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use BayeSQP instead of TuRBO | Ackley5D · Ackley5D | objective value: 8.95 vs 0.77 | Hurt |
| Use BayeSQP instead of TuRBO | Hartmann · Hartmann | objective value: -3.30 vs -3.32 | Hurt |
| Use BayeSQP instead of TuRBO | Ackley20D · Ackley20D | objective value: 10.66 vs 2.20 | Hurt |
| Use BayeSQP instead of SCBO | Ackley5D (constr.) · Ackley5D (constr.) | objective value: 6.25 vs 0.51 | Hurt |
| Use BayeSQP instead of SCBO | Hartmann (constr.) · Hartmann (constr.) | objective value: -3.32 vs -3.32 | No clear effect |
| Use BayeSQP instead of SCBO | Ackley20D (constr.) · Ackley20D (constr.) | objective value: 3.90 vs 2.03 | Hurt |
| Use BayeSQP instead of C-logEI | Speed Reducer · Speed Reducer | performance: 3002.81 -> 3001.10 | Helped |
| Use BayeSQP instead of C-logEI | Speed Reducer · Speed Reducer | Avg. runtime (s): 3464.59 -> 91.83 | Helped |
| Use BayeSQP instead of TuRBO | within-model optimization · high-dimensional problems | performance | Helped |
| Reduce the number of sub-samples K below d (ablation) | BayeSQP · constrained within-model | performance | Helped |
| Increase K beyond d (ablation) | BayeSQP · constrained within-model | performance | Hurt |
| Increase the number of line-search samples M (ablation) | BayeSQP · out-of-model comparisons | performance | Helped |
Surprising findings
- Local BayeSQP performs worse than global baselines on multimodal Ackley (e.g., 8.95 vs 0.77 for TuRBO on Ackley5D), showing sensitivity to initialization.
- Reducing the number of subsampling points K below d can improve performance, suggesting fewer subsampling steps with more subproblem solves may be beneficial.
Limitations
- Performance depends on initialization and converges to different local optima (e.g., Gramacy), so it is a local method.
- Full Hessian computation and storage scale poorly with dimension and number of constraints.
- Sensitive to kernel and model assumptions; infeasible subproblems require slack fallback.
Experimental setup
Benchmarks: Ackley5D Hartmann Ackley20D Speed Reducer Gramacy
Metrics: best objective value feasibility rate optimization runtime
Baselines: logEI C-logEI TuRBO SCBO SAASBO MPD
Builds on: GIBO sequential quadratic programming LineBO SCBO
Closest papers by meaning
- Efficient Nonmyopic Bayesian Optimization via One-Shot Multi-Step Trees (NeurIPS 2020 · similarity 0.90)
- Minimizing UCB: a Better Local Search Strategy in Local Bayesian Optimization (NeurIPS 2024 · similarity 0.88)
- MOBO-OSD: Batch Multi-Objective Bayesian Optimization via Orthogonal Search Directions (NeurIPS 2025 · similarity 0.87)
- BARK: A Fully Bayesian Tree Kernel for Black-box Optimization (ICML 2025 · similarity 0.87)
- Deep Distributed Optimization for Large-Scale Quadratic Programming (ICLR 2025 · similarity 0.87)
- Understanding High-Dimensional Bayesian Optimization (ICML 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.