Research map / Probabilistic and scientific modeling / Bayesian optimization
Optimizing the Unknown: Black Box Bayesian Optimization with Energy-Based Model and Reinforcement Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes REBMBO, which combines a GP surrogate, a short-run-MCMC-trained energy-based model (EBM-UCB acquisition), and PPO-based multi-step planning, framing each BO iteration as an MDP, and reports gains over BO baselines on synthetic and real-world benchmarks.
Problem
Classic Bayesian optimization is one-step myopic and tends to converge to local optima on complex, high-dimensional, multi-modal black-box functions under limited evaluation budgets.
Approach
Adds an energy-based global term (−γE_θ(x)) to a GP-UCB acquisition, treats each BO iteration as an MDP solved with PPO for multi-step lookahead, and defines a Landscape-Aware Regret metric that adds an energy-based penalty to standard regret.
Key results
- REBMBO-C reaches 97.37±2.07 on Branin 2D at T=50 vs 90.44±1.98 for BALLET-ICI (score table, higher is better).
- REBMBO-D achieves mean 86.98±2.80 on real-world benchmarks vs 84.39±2.74 for KG (best baseline) in the real-world table.
- Paper claims REBMBO-S achieves roughly 15–20% lower final pseudo-regret than EARL-BO and BALLET-ICI on Branin and Ackley.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add EBM global exploration signals (ablation) | REBMBO · Branin 2D | final objective: 8.95 \(\pm\) 0.20 -> 9.30 \(\pm\) 0.12 | Helped |
| Add EBM global exploration signals (ablation) | REBMBO · Nanophotonic 3D | final objective: -1.05 \(\pm\) 0.07 -> -0.85 \(\pm\) 0.04 | Helped |
| Add multi-step PPO lookahead (ablation) | REBMBO · Branin 2D | final objective: 9.10 \(\pm\) 0.15 -> 9.30 \(\pm\) 0.12 | Helped |
| Add multi-step PPO lookahead (ablation) | REBMBO · Nanophotonic 3D | final objective: -0.92 \(\pm\) 0.05 -> -0.85 \(\pm\) 0.04 | Helped |
| Train the EBM with short-run MCMC (ablation) | REBMBO · Branin 2D | final objective: 9.00 \(\pm\) 0.18 -> 9.30 \(\pm\) 0.12 | Helped |
| Train the EBM with short-run MCMC (ablation) | REBMBO · Nanophotonic 3D | final objective: -0.97 \(\pm\) 0.06 -> -0.85 \(\pm\) 0.04 | Helped |
| Keep PPO and fully train the EBM rather than removing PPO and using incomplete EBM training (ablation) | REBMBO · Branin 2D | final objective: 8.60 \(\pm\) 0.28 -> 9.30 \(\pm\) 0.12 | Helped |
| Keep PPO and fully train the EBM rather than removing PPO and using incomplete EBM training (ablation) | REBMBO · Nanophotonic 3D | final objective: -1.02 \(\pm\) 0.06 -> -0.85 \(\pm\) 0.04 | Helped |
| Use REBMBO-C instead of GP-UCB | Bayesian optimization · Sphere (2D) | regret improvement (%): -8.5 | Hurt |
| Use REBMBO-C instead of GP-UCB | Bayesian optimization · Ackley (20D) | regret improvement (%): 46.7 | Helped |
Surprising findings
- Reported ablation shows Model B (no PPO) still outperforms a purely local baseline, and the paper's own tables show REBMBO-D losing to REBMBO-C on some real-world tasks such as Nanophotonic 3D.
Limitations
- Theoretical convergence analysis under RL and approximate EBM training is left to future work, as the conclusion acknowledges.
- Reported results are averaged over only 5 runs, and some reported numbers (e.g., score-style tables) are not clearly defined relative to regret.
Experimental setup
Datasets: Branin Ackley Rosenbrock HDBO 200D synthetic Nanophotonic Rosetta protein design NATS-Bench Robot Trajectory
Benchmarks: Branin 2D Ackley 5D Rosenbrock 8D HDBO 200D Nanophotonic 3D Rosetta 86D NATS-Bench 20D Robot Trajectory 40D
Metrics: Landscape-Aware Regret normalized objective score standard regret
Baselines: Classic BO BALLET-ICI TuRBO EARL-BO Two-step EI KG GP-UCB GLASSES
Builds on: Gaussian Process Bayesian optimization GP-UCB TuRBO EARL-BO Proximal Policy Optimization
Closest papers by meaning
- Augmented Bayesian Policy Search (ICLR 2024 · similarity 0.90)
- Transition Constrained Bayesian Optimization via Markov Decision Processes (NeurIPS 2024 · similarity 0.90)
- A General Framework for User-Guided Bayesian Optimization (ICLR 2024 · similarity 0.88)
- Informed Initialization for Bayesian Optimization and Active Learning (NeurIPS 2025 · similarity 0.88)
- Batched Energy-Entropy acquisition for Bayesian Optimization (NeurIPS 2024 · similarity 0.88)
- Boundary Exploration for Bayesian Optimization With Unknown Physical Constraints (ICML 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.