Research map / Probabilistic and scientific modeling / Variational and generative sampling
Variational Search Distributions
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Variational Search Distributions (VSD), a variational-inference method for active generation that learns a generative model of rare, desirable discrete designs via batch sequential black-box evaluation, with convergence guarantees and experiments on protein/DNA sequence tasks.
Problem
Generating batches of rare, feasible designs in a combinatorially large discrete space under a fixed black-box evaluation budget, while learning a generative model of the super level-set distribution.
Approach
Cast active generation as variational inference minimizing reverse KL to the level-set posterior, optimized via REINFORCE with an acquisition-weighted ELBO using a class probability estimator or GP-PI, sampling each round from the learned variational distribution rather than selecting from a candidate list.
Key results
- On the combinatorially near-complete fitness landscapes (DHFR, TrpB, TFBIND8), VSD is the best performing method by most precision, recall and performance measures.
- On AAV and GFP black-box optimization, VSD is among the leading methods, while AdaLead, PEX and BORE often perform worse than random.
- On Ehrlich functions, VSD outperforms CBAS and is competitive with LaMBO-2.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use VSD instead of the competing methods | Active generation on DHFR, TrpB, and TFBIND8 · DHFR, TrpB, and TFBIND8 | Precision, recall, and performance | Helped |
| Use an informative prior instead of a uniform prior (ablation) | VSD · AAV and GFP | Black-box optimization performance | Helped |
| Use an LSTM or transformer autoregressive prior and posterior instead of independent or transition distributions (ablation) | VSD · AAV and GFP | Black-box optimization performance | Helped |
| Use an independent or transition variational distribution instead of an autoregressive distribution (ablation) | VSD · AAV and GFP | Black-box optimization performance | Hurt |
| Replace an independent variational distribution with a transition-style autoencoder posterior (ablation) | VSD · AAV and GFP | Black-box optimization performance | No clear effect |
| Replace an independent variational distribution with a transition-style convolutional autoencoder posterior (ablation) | VSD · AAV and GFP | Black-box optimization performance | No clear effect |
| Use a uniform prior instead of an informed prior (ablation) | VSD · Higher-dimensional black-box optimization | Optimization progress within the experimental budget | Hurt |
| Use an autoregressive variational distribution instead of a mean-field distribution (ablation) | VSD · Ehrlich functions | Simple regret | Helped |
| Use VSD instead of CBAS | Ehrlich function benchmarks · Ehrlich functions | Simple regret | Helped |
| Use VSD instead of LaMBO-2 | Ehrlich function benchmarks · Ehrlich functions | Simple regret | Mixed |
| Use the GP-PI acquisition estimator instead of CPE-PI (ablation) | VSD · DHFR, TrpB, and TFBIND8 | Precision, recall, and performance | Mixed |
Surprising findings
- The adapted BORE* collapses to a point-mass variational distribution and performs poorly in batch settings, showing the value of KL regularization.
- AdaLead and PEX, evolutionary search methods, degrade in performance as sequence dimension increases.
Limitations
- Relies on informed priors for high-dimensional problems; uniform priors fail on AAV/GFP.
- Oracles for AAV and GFP are CNN predictors that may not extrapolate out-of-distribution.
- Theoretical rates hold under assumptions and batch size B=1, with batch results only heuristic.
Experimental setup
Datasets: MNIST DHFR TrpB TFBIND8 AAV GFP Ehrlich functions
Benchmarks: DHFR TrpB TFBIND8 AAV GFP Ehrlich functions (poli) Ehrlich functions (holo) MNIST digits 3 and 5
Metrics: precision recall performance simple regret diversity (Levenshtein) balanced accuracy
Baselines: DBAS CBAS AdaLead PEX BORE LaMBO-2 genetic algorithm random sampling
Builds on: Bayesian optimization CBAS DBAS BORE Adam
Closest papers by meaning
- A Variational Perspective on Generative Protein Fitness Optimization (ICML 2025 · similarity 0.88)
- Variable-Length Generative Protein Design via Generalized Poisson Flow (NeurIPS 2026 · similarity 0.86)
- Bridging Model-Based Optimization and Generative Modeling via Conservative Fine-Tuning of Diffusion Models (NeurIPS 2024 · similarity 0.86)
- ProSpero: Active Learning for Robust Protein Design Beyond Wild-Type Neighborhoods (NeurIPS 2025 · similarity 0.86)
- Flow Density Control: Generative Optimization Beyond Entropy-Regularized Fine-Tuning (NeurIPS 2025 · similarity 0.86)
- Generative Adversarial Model-Based Optimization via Source Critic Regularization (NeurIPS 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.