Research map / Image generation / Autoregressive and latent generation
Absorb & Escape: Overcoming Single Model Limitations in Generating Heterogeneous Genomic Sequences
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Absorb & Escape (A&E), a post-training sampler that composes an autoregressive DNA model (HyenaDNA) with a latent discrete diffusion model (DiscDiff) via a fast heuristic variant, to generate heterogeneous genomic promoter sequences across 15 species.
Problem
Single generative models (AR or diffusion) struggle to generate heterogeneous genomic sequences whose functional regions follow different distributions; composing models usually requires slow MCMC sampling.
Approach
Start from a diffusion model sample and iteratively refine low-confidence segments with an autoregressive model via Absorb (AR resampling of a segment) and Escape (accept the refinement) steps; Fast A&E thresholds per-token DM probabilities so that at most one AR pass per position is needed, avoiding MCMC.
Key results
- On TP-conditioned promoter design, A&E (Language Model + DFM distilled) achieved MSE 0.0262, the lowest among the listed methods (DFM 0.0269).
- In the toy HMM study, DiscDiff produced 0 illegal start tokens but 110,192 incorrect transition tokens, versus HyenaDNA's 812 and 3,586.
- In Enformer genome integration, A&E gave the lowest SSE for TP53 (17.21 vs Hyena 36.25, DiscDiff 124.03).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| combine the Language Model and distilled Dirichlet FM with Fast A&E sampling | TP-conditioned promoter design · TP-conditioned promoter design test split | MSE: .0278 -> .0262 | Helped |
| use A&E instead of Hyena | promoter genome integration · TP53 | SSE: 36.25 -> 17.21 | Helped |
| use A&E instead of DiscDiff | promoter genome integration · TP53 | SSE: 124.03 -> 17.21 | Helped |
| use A&E instead of Hyena | promoter genome integration · EGFR | SSE: 0.89 -> 0.28 | Helped |
| use A&E instead of DiscDiff | promoter genome integration · EGFR | SSE: 2.17 -> 0.28 | Helped |
| use A&E instead of Hyena | promoter genome integration · AKT1 | SSE: 2.88 -> 1.65 | Helped |
| use A&E instead of DiscDiff | promoter genome integration · AKT1 | SSE: 25.50 -> 1.65 | Helped |
| combine a diffusion model initialization with autoregressive correction using Fast A&E | multi-species promoter generation · 15 species; TATA-box, GC-box, Initiator, and CCAAT-box motif distributions | motif distribution MSE | Helped |
Surprising findings
- Most existing diffusion models (D3PM, BitDiffusion, DDSM) underperform the simple DiscDiff baseline on TATA-box metrics, indicating a gap in current diffusion DNA models.
- The toy experiment shows the DM and AR models make complementary error types, with the DM producing many incorrect transitions despite zero illegal start tokens.
Limitations
- Convergence proof relies on strong assumptions (accurate AR and DM approximations, homogeneous segments, ergodicity) that are not verified in practice.
- Results on generated sequences are not validated by wet-lab experiments; motif and Enformer proxies only.
- Fast A&E's threshold T_absorb must be tuned on validation data; the paper uses different defaults (0.85 in text, 0.80 in appendix).
Experimental setup
Datasets: Eukaryotic Promoter Database (EPDnew) subset, 160K sequences across 15 species Sei training data for TP-conditioned promoter design (DFM)
Benchmarks: TP-conditioned promoter design (MSE via Sei) multi-species promoter motif distribution (EPD, 15 species) unconditional EPD 256bp/2048bp (S-FID, CorTATA) BLASTN sequence diversity Enformer genome integration (SSE) on TP53, EGFR, AKT1
Metrics: MSE S-FID motif correlation motif MSE BLASTN score/eValue/AlignLen SSE of transcription profiles illegal start and incorrect transition token counts
Baselines: DDSM DNADiffusion D3PM Bit Diffusion Dirichlet FM HyenaDNA DiscDiff VAE
Builds on: Gibbs sampling Stable Diffusion (latent diffusion) HyenaDNA DDSM Dirichlet flow matching
Closest papers by meaning
- Diffusion on Language Model Encodings for Protein Sequence Generation (ICML 2025 · similarity 0.86)
- Fine-Tuning Discrete Diffusion Models via Reward Optimization with Applications to DNA and Protein Design (ICLR 2025 · similarity 0.85)
- Dirichlet Flow Matching with Applications to DNA Sequence Design (ICML 2024 · similarity 0.85)
- Bridging Model-Based Optimization and Generative Modeling via Conservative Fine-Tuning of Diffusion Models (NeurIPS 2024 · similarity 0.85)
- Steering Generative Models with Experimental Data for Protein Fitness Optimization (NeurIPS 2025 · similarity 0.85)
- JanusDNA: A Powerful Bi-directional Hybrid DNA Foundation Model (NeurIPS 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.