Research map / Probabilistic and scientific modeling / Bayesian optimization
Inversion-based Latent Bayesian Optimization
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes InvBO, a plug-and-play module for VAE-based latent Bayesian optimization that uses optimization-based inversion to align encoder-decoder data without extra oracle calls and a potential-aware trust region anchor selection.
Problem
Latent Bayesian optimization over discrete structures suffers from a misalignment between latent codes and inputs caused by VAE reconstruction error, which yields inaccurate surrogates and poor solutions; trust-region anchors are also chosen only by objective value.
Approach
Invert the frozen decoder by optimizing the latent code so that the decoded output reconstructs the target discrete data (building aligned decoder triplets without extra oracle calls), and pick trust-region anchors by adding a Thompson-sampling-based potential score to the observed objective value.
Key results
- On valt with 500 oracle calls, CoBO+InvBO reaches 0.348 versus 0.000 for vanilla CoBO, and LOL-BO+InvBO reaches 0.171 versus 0.000.
- On med2 with 500 oracle calls, LOL-BO+InvBO improves the best score from 0.190 to 0.227.
- Inversion reduces the surrogate's fitting error and yields zero dissimilarity between x and decoded p(z) at every iteration, versus mostly misaligned data without inversion.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add InvBO to trust-region LBO methods | TuRBO-L, LOL-BO, and CoBO · med2 and valt | optimization score: 0.191$\pm$0.003 -> 0.214$\pm$0.006; 0.000$\pm$0.000 -> 0.348$\pm$0.107 | Helped |
| Add inversion to LBO methods | LBO, W-LBO, and PG-LBO · Guacamol tasks | optimization score: 0.502$\pm$0.006 -> 0.526$\pm$0.002 | Helped |
| Add inversion to the dataset construction and VAE-update process (ablation) | CoBO · osmb | average best score: 0.784, 0.792, and 0.804 score gains compared to vanilla CoBO | Helped |
| Add potential-aware trust region anchor selection (ablation) | CoBO · osmb | average best score: 0.784, 0.792, and 0.804 score gains compared to vanilla CoBO | Helped |
| Use decoder triplets generated by inversion instead of encoder triplets (ablation) | CoBO with potential-aware anchor selection · valt and med2 | optimization score | Helped |
| Use inversion to align training data (ablation) | LBO · med2 and valt | normalized Levenshtein distance | Helped |
| Use potential-aware anchor selection instead of objective-score-only selection (ablation) | LBO with inversion · valt and med2 | optimization score | Helped |
| Use potential-aware anchor selection instead of objective-score-only selection (ablation) | CoBO + InvBO · two tasks | number of searched unique data samples | Helped |
| Add CoBO regularization losses (ablation) | LBO surrogate model · prediction-error analysis | objective value prediction error | Helped |
| Combine CoBO regularization losses with inversion (ablation) | CoBO + InvBO · prediction-error analysis | objective value prediction error | Helped |
Surprising findings
- Prior recentering techniques in LOL-BO and CoBO consume additional oracle calls not stated in the papers; with recentering disabled, CoBO fails on med2 and valt.
- Vanilla CoBO and LOL-BO achieve zero score on valt within 500 oracle calls, which InvBO substantially improves.
Limitations
- Performance depends on the quality of the generative model (VAE) and its ability to produce valid molecules.
- The inversion relies on a task-specific distance function; the paper uses normalized Levenshtein distance by default.
- Theoretical result relies on Lipschitz assumptions and the bound is loose in practice.
Experimental setup
Datasets: Guacamol DRD3 arithmetic expression fitting
Benchmarks: Guacamol (med2, zale, pdop, adip, osmb, rano, valt) DRD3 arithmetic expression fitting
Metrics: best objective score under oracle budget wall-clock time
Baselines: LS-BO TuRBO-L W-LBO LOL-BO CoBO PG-LBO GB-GA
Builds on: TuRBO LOL-BO CoBO GrammarVAE SELFIES VAE
Closest papers by meaning
- Latent Bayesian Optimization via Autoregressive Normalizing Flows (ICLR 2025 · similarity 0.87)
- PG-LBO: Enhancing High-Dimensional Bayesian Optimization with Pseudo-Label and Gaussian Process Guidance (AAAI 2024 · similarity 0.87)
- A Study of Bayesian Neural Network Surrogates for Bayesian Optimization (ICLR 2024 · similarity 0.86)
- Joint Composite Latent Space Bayesian Optimization (ICML 2024 · similarity 0.86)
- Informed Initialization for Bayesian Optimization and Active Learning (NeurIPS 2025 · similarity 0.86)
- ROOT: Rethinking Offline Optimization as Distributional Translation via Probabilistic Bridge (NeurIPS 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.