Research map / Probabilistic and scientific modeling / Variational and generative sampling
Learning Extrapolative Sequence Transformations from Markov Chains
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Learns an autoregressive extrapolation model from selected states of MCMC chains (Metropolis-Hastings with a mask-infill proposal), so that few-step generation extrapolates beyond training score ranges on protein stability, sentiment, and anonymization tasks.
Problem
Generating sequences that improve a property beyond the range seen in training data (extrapolation) is hard; MCMC is sample-inefficient and test-time search is slow, so an efficient model that greedily extrapolates in few steps is needed.
Approach
Use Metropolis-Hastings chains with a masked-LM (infilling) proposal as a source of training episodes, select state transitions that improve the energy (e.g., first/best or delta-energy thinning), and fine-tune an autoregressive model to iteratively transform a sequence over a few intermediate states conditioned on scores, amortizing the MCMC search.
Key results
- On ACE2, q_theta reaches 0.748 at ddG<-5 and 0.464 at ddG<-7 versus 0.270 and 0.005 for MCMC, using 3 iterations vs 83.
- On sentiment, q_theta (first/best) gets 0.734 extrapolation with a 0.132% fluency drop and 1 iteration, versus MCMC's 0.809 extrapolation with 0.746% drop and 496 iterations.
- On anonymization, q_theta reaches SBERT 0.839 and EER 0.221 in 4 iterations, while MCMC reaches EER 0.393 and SBERT 0.835 in 4498 iterations.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| train on fixed-length Δ-energy episodes rather than first/best episodes (ablation) | qθ · ACE2 stability | proportion ddG < -5: 0.609 -> 0.748 | Helped |
| train on variable-length Δ-energy episodes rather than fixed-length Δ-energy episodes (ablation) | qθ · ACE2 stability | proportion ddG < -5: 0.748 -> 0.424 | Hurt |
| use Δ-energy episode selection rather than thinning (ablation) | qθ · Sentiment extrapolation | extrapolation: 0.642 -> 0.692; 0.591 -> 0.677 | Mixed |
| use first/best episodes rather than fixed-length thinning (ablation) | qθ · Sentiment extrapolation | extrapolation: 0.642 -> 0.734 | Helped |
| use variable-length Δ-energy episodes rather than first/best episodes (ablation) | qθ · Anonymization | EER: 0.132 -> 0.221 | Helped |
| use variable-length Δ-energy episodes rather than first/best episodes (ablation) | qθ · Anonymization | SBERT: 0.923 -> 0.839 | Hurt |
| predict rewards rather than use real rewards (ablation) | qθ · Anonymization | EER: 0.179 -> 0.221 | Helped |
| predict rewards rather than use real rewards (ablation) | qθ · Anonymization | SBERT: 0.693 -> 0.839 | Helped |
| use predicted reward rather than real reward (ablation) | qθ · Sentiment extrapolation | extrapolation: 0.679 -> 0.692; 0.474 -> 0.677 | Mixed |
| train on the full MCMC chain rather than the first 25% of it (ablation) | qθ · Anonymization | SBERT: 0.701 -> 0.835 | Helped |
| train on the full MCMC chain rather than the first 25% of it (ablation) | qθ · Anonymization | EER: 0.200 -> 0.224 | Helped |
| use qθ rather than MCMC | ACE2 stability · ACE2 stability | proportion ddG < -7: 0.005 -> 0.464 | Helped |
Surprising findings
- Further MCMC sampling (ten epochs) did not approach the performance of the learned model on protein engineering, suggesting a generalization benefit beyond sampling.
- Variable-length delta-energy episodes performed much worse than fixed-length on protein engineering, though not on sentiment or anonymization.
Limitations
- Extrapolation baselines are limited; prompting-based LLM baselines were not compared on extrapolation tasks.
- Generating synthetic training data via MCMC remains computationally expensive.
- Protein results depend on a learned ddG scorer and FoldX evaluation, and the MCMC baseline did not match published baselines on the protein task.
Experimental setup
Datasets: ACE2 stability dataset Yelp reviews Reddit IUR dataset
Benchmarks: ACE2 stability (FoldX ddG) Yelp sentiment extrapolation Reddit IUR anonymization
Metrics: proportion below ddG thresholds training and extrapolation range proportions median GPT-2 perplexity change EER SBERT similarity iterations
Baselines: Mask/Infill Iterative sampling Genhance ICE MCMC FUDGE GPT-3.5 GPT-4 DIPPER Round Trip MT
Builds on: Iterative Controllable Extrapolation Metropolis-Hastings decision transformer FUDGE
Closest papers by meaning
- Extrapolative Controlled Sequence Generation via Iterative Refinement (ICML 2023 · similarity 0.88)
- AutoStep: Locally adaptive involutive MCMC (ICML 2025 · similarity 0.84)
- Steering Masked Discrete Diffusion Models via Discrete Denoising Posterior Prediction (ICLR 2025 · similarity 0.84)
- Fine-Tuning Discrete Diffusion Models via Reward Optimization with Applications to DNA and Protein Design (ICLR 2025 · similarity 0.84)
- Curriculum Learning for Biological Sequence Prediction: The Case of De Novo Peptide Sequencing (ICML 2025 · similarity 0.84)
- Parallelizing MCMC Across the Sequence Length (NeurIPS 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.