Research map / Probabilistic and scientific modeling / Bayesian optimization
Latent Bayesian Optimization via Autoregressive Normalizing Flows
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes NF-BO, which uses a sequence-specialized autoregressive normalizing flow (SeqFlow) for invertible latent encoding to eliminate the VAE reconstruction gap in latent Bayesian optimization, plus token-level adaptive candidate sampling (TACS) for molecule design.
Problem
Latent Bayesian optimization over discrete sequences (e.g., SELFIES molecules) suffers from a value discrepancy problem: VAE reconstructions do not preserve objective values, so errors propagate through the optimization.
Approach
Replace the VAE with an autoregressive normalizing flow that maps discrete token embeddings to a latent space via an injective embedding lookup (argmax cosine similarity) and an invertible flow, giving exact left-inverse reconstruction; sample local-search candidates by perturbing latent tokens according to PMI-based token importance.
Key results
- On PMO (23 tasks), NF-BO ranks 1st on five of six metrics, with a Top-1 sum of 18.095 versus 16.772 for the next best (REINVENT SMILES).
- NF-BO improves VAE BO with SELFIES from rank 19 to rank 1 on PMO Top-1 when using SeqFlow.
- SeqFlow reports zero value discrepancy (y != y_hat ratio 0.000) on seven GuacaMol tasks, versus 0.5-0.8 for TextFlow on most tasks.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use NF-BO with SeqFlow instead of VAE BO | PMO benchmarks, 23 tasks · PMO | average rank: 19th to 1st | Helped |
| Add TACS to NF-BO (ablation) | Guacamol adip, init 1 · Guacamol adip, init 1 | optimization score: 0.765 ± 0.038 -> 0.818 ± 0.051 | Helped |
| Add TACS to NF-BO (ablation) | Guacamol med2, init 1 · Guacamol med2, init 1 | optimization score: 0.306 ± 0.014 -> 0.307 ± 0.027 | No clear effect |
| Use SeqFlow's exact-reconstruction mapping instead of BiLSTM mapping (ablation) | NF-BO w/o TACS · Guacamol adip | optimization score: 0.716 ± 0.017 -> 0.778 ± 0.016 | Helped |
| Use SeqFlow's exact-reconstruction mapping instead of BiLSTM mapping (ablation) | NF-BO w/o TACS · Guacamol med2 | optimization score: 0.347 ± 0.010 -> 0.372 ± 0.012 | Helped |
| Set TACS temperature to 200 instead of removing TACS (ablation) | NF-BO · Guacamol, 7 tasks | score sum: 5.495 -> 5.565 | Helped |
| Set TACS temperature to 100 instead of removing TACS (ablation) | NF-BO · Guacamol, 7 tasks | score sum: 5.495 -> 5.481 | Hurt |
| Set TACS temperature to 400 instead of removing TACS (ablation) | NF-BO · Guacamol, 7 tasks | score sum: 5.495 -> 5.544 | Helped |
| Replace uniform token perturbation with TACS (ablation) | NF-BO · Guacamol benchmarks | optimization performance | Helped |
Surprising findings
- TextFlow's BiLSTM mapping yields value-discrepancy ratios of 0.5-0.8 on most GuacaMol tasks, showing that non-injective mappings break reconstruction despite the flow being invertible.
- With only one initial data point, NF-BO with TACS sometimes outperforms its 10K-initial-data variant (e.g., Adip), suggesting TACS helps under low-data conditions.
Experimental setup
Datasets: GuacaMol ZINC PMO benchmark
Benchmarks: GuacaMol PMO
Metrics: top-1 score top-10 score top-100 score AUC top-1 AUC top-10 AUC top-100 sum of scores
Baselines: LSBO TuRBO-L W-LBO LOLBO CoBO PG-LBO GP BO VAE BO JT-VAE BO REINVENT Graph GA STONED
Builds on: latent bayesian optimization normalizing flows TuRBO LOLBO TextFlow
Closest papers by meaning
- PG-LBO: Enhancing High-Dimensional Bayesian Optimization with Pseudo-Label and Gaussian Process Guidance (AAAI 2024 · similarity 0.88)
- Model-Informed Flows for Bayesian Inference (NeurIPS 2025 · similarity 0.88)
- Annealing Flow Generative Models Towards Sampling High-Dimensional and Multi-Modal Distributions (ICML 2025 · similarity 0.88)
- Posterior Inference in Latent Space for Scalable Constrained Black-box Optimization (NeurIPS 2026 · similarity 0.87)
- Joint Composite Latent Space Bayesian Optimization (ICML 2024 · similarity 0.87)
- Inversion-based Latent Bayesian Optimization (NeurIPS 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.