Research map / Probabilistic and scientific modeling / Bayesian optimization

Posterior Inference in Latent Space for Scalable Constrained Black-box Optimization

NeurIPS 2026 · Probabilistic and scientific modeling › Bayesian optimization · approach: Offline black-box optimization with surrogates · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes CiBO, which casts constrained high-dimensional black-box candidate selection as latent-space posterior inference, using a flow-model prior, ensemble/constraint surrogates, and an outsourced TB-trained diffusion sampler.

Problem

Constrained black-box optimization in high dimensions, where feasible regions are hard to find and the candidate posterior is highly multi-modal with flat regions, making sampling and fine-tuning generative models hard.

Approach

Amortize posterior sampling p_post(x) ∝ p_θ(x)exp(β r_φ(x)) in the latent space of a flow-based prior using an outsourced diffusion sampler trained with trajectory balance, combined with rank-based reweighted training of the flow and surrogates using a Lagrangian score.

Key results

• CiBO outperforms all baselines on the three 200D synthetic tasks under both black-box and indicator constraint settings, per the paper's synthetic experiments.
• On real-world tasks, DiffOPT achieves feasibility only on HalfCheetah, and SCBO is the only other baseline reaching feasibility on most tasks but with low sample efficiency.
• CiBO reaches near-perfect feasibility within 5-10 batches on Rastrigin-200D, roughly twice as fast as BO baselines (SCBO, PCAGP-SCBO).

Surprising findings

• Mode-collapse-prone fine-tuning is avoided by sampling in latent space, where the posterior is reported to be smoother than in data space.
• Simply adapting unconstrained generative methods (DDOM, DiBO) with Lagrangian relaxation underperforms CiBO, suggesting constraint-aware design is needed.

Limitations

• Requires retraining all models (flow, surrogates, diffusion sampler) from scratch every round, which is costly.
• Performance is sensitive to surrogate accuracy when beta is too high; lambda and beta require task-specific tuning.
• Rastrigin and Ackley constraint centers coincide with optima in the main setup, which may favor methods; misaligned variant is only partially explored.

Experimental setup

Datasets: Rastrigin 200D Ackley 200D Rosenbrock 200D RoverPlanning 60D HalfCheetah 102D Mopta 124D (MOPTA08) LassoBench DNA 180D

Benchmarks: Rastrigin 200D Ackley 200D Rosenbrock 200D RoverPlanning 60D HalfCheetah 102D Mopta 124D LassoBench DNA 180D

Metrics: best feasible objective value found feasibility ratio per-round wall-clock time

Baselines: cEI LogcEI SCBO PCAGP-SCBO FuRBO DiffOPT CMA-ES COBYLA Sobol DDOM DiBO

Builds on: outsourced diffusion sampling trajectory balance flow matching DiffOPT SCBO

Closest papers by meaning

• Posterior Inference with Diffusion Models for High-dimensional Black-box Optimization (ICML 2025 · similarity 0.93)
• ROOT: Rethinking Offline Optimization as Distributional Translation via Probabilistic Bridge (NeurIPS 2025 · similarity 0.88)
• Outsourced Diffusion Sampling: Efficient Posterior Inference in Latent Spaces of Generative Models (ICML 2025 · similarity 0.87)
• Batch and match: black-box variational inference with a score-based divergence (ICML 2024 · similarity 0.87)
• Latent Bayesian Optimization via Autoregressive Normalizing Flows (ICLR 2025 · similarity 0.87)
• Denoising Diffusion Variational Inference: Diffusion Models as Expressive Variational Posteriors (AAAI 2025 · similarity 0.87)