Research map / Reinforcement learning / Visual and representation RL
Regulatory DNA Sequence Design with Reinforcement Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes TACO, which fine-tunes a pre-trained HyenaDNA autoregressive model with REINFORCE using a fitness oracle plus SHAP-derived TFBS rewards to design high-fitness, diverse yeast promoters and human enhancers.
Problem
Designing high-fitness cis-regulatory elements (promoters and enhancers) is hard: iterative mutation-based optimizers get stuck in local optima, produce low diversity, and ignore biological priors about transcription factor binding sites.
Approach
Treat a pre-trained autoregressive DNA model as an RL policy, generate sequences from scratch, and add dense per-token rewards for activator TFBS insertion and repressor TFBS removal, where TFBS roles are inferred from SHAP values of a LightGBM model trained on TFBS frequency features.
Key results
- On the yeast promoter hard subsets, TACO reaches Top 1 and Medium 1 with Diversity 52.8 (complex) and 49.6 (defined), above the next baseline's diversity (PEX 9.8 and 10.2 respectively).
- On human enhancers, TACO attains the best Top and Medium for K562 and SK-N-SH (e.g., K562-hard Top 0.93, Medium 0.91) with diversity about 125-137, while PEX achieves higher fitness on HepG2 with diversity below 20.
- In offline MBO on K562, TACO reaches Top 0.75 and Medium 0.72 with Diversity 102.6, comparable to PEX (Top 0.76) and higher diversity than all optimization baselines except regLM.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add CRE pre-training to the policy (ablation) | TACO · HepG2 hard | Medium: 0.55 -> 0.60 | Helped |
| Add CRE pre-training to the policy (ablation) | TACO · K562 hard | Medium: 0.59 -> 0.72 | Helped |
| Add CRE pre-training to the policy (ablation) | TACO · SK-N-SH hard | Medium: 0.57 -> 0.62 | Helped |
| Add TFBS reward (ablation) | TACO · HepG2 hard | Medium: 0.58 -> 0.60 | Helped |
| Add TFBS reward (ablation) | TACO · K562 hard | Medium: 0.71 -> 0.72 | Helped |
| Add TFBS reward (ablation) | TACO · SK-N-SH hard | Medium: 0.60 -> 0.62 | Helped |
| Increase α from 0.01 to 0.1 (ablation) | TACO · K562 hard | Medium: 0.72 -> 0.77 | Helped |
| Increase α from 0.01 to 0.1 (ablation) | TACO · K562 hard | Diversity: 102.6 (20.14) -> 82.8 (4.02) | Hurt |
| Increase α from 0.01 to 0.1 (ablation) | TACO · SK-N-SH hard | Medium: 0.62 -> 0.65 | Helped |
| Increase α from 0.01 to 0.1 (ablation) | TACO · HepG2 hard | Medium: 0.60 -> 0.58 | Hurt |
| Fine-tune HyenaDNA on short CRE sequences (ablation) | HyenaDNA · HepG2 hard | Top: 0.749 -> 0.751 | Helped |
| Fine-tune HyenaDNA on short CRE sequences (ablation) | HyenaDNA · HepG2 hard | Medium: 0.723 -> 0.729 | Helped |
Surprising findings
- A LightGBM model using only TFBS frequency features reaches Pearson 0.65 on HepG2, versus 0.83 for Enformer using sequence input.
- Increasing alpha to 0.1 improves Top and Medium for K562 and SK-N-SH but sharply reduces diversity, while all metrics on HepG2 worsen.
Limitations
- TFBS candidates come from a fixed database (JASPAR), bounding the TFBS reward.
- TF roles are inferred from TFBS frequency only, ignoring TF-TF interactions and orientation.
- No established validity criteria for generated DNA sequences; novelty metric omitted.
Experimental setup
Datasets: yeast promoter complex and defined MPRA datasets human enhancer MPRA datasets HepG2, K562, SK-N-SH JASPAR TFBS database
Benchmarks: yeast promoter (complex, defined) human enhancer (HepG2, K562, SK-N-SH)
Metrics: Top Medium Diversity Emb Similarity
Baselines: PEX AdaLead BO CMAES DNARL regLM DDSM
Builds on: HyenaDNA REINFORCE regLM LatProtRL SHAP
Closest papers by meaning
- Designing Cell-Type-Specific Promoter Sequences Using Conservative Model-Based Optimization (NeurIPS 2024 · similarity 0.85)
- Designing Biological Sequences without Prior Knowledge Using Evolutionary Reinforcement Learning (AAAI 2024 · similarity 0.85)
- Learning to Discover Regulatory Elements for Gene Expression Prediction (ICLR 2025 · similarity 0.84)
- Improved Off-policy Reinforcement Learning in Biological Sequence Design (ICML 2025 · similarity 0.84)
- C3P: Contrastive promoter-protein pretraining yields representations capturing bacterial gene regulation (NeurIPS 2026 · similarity 0.83)
- Absorb & Escape: Overcoming Single Model Limitations in Generating Heterogeneous Genomic Sequences (NeurIPS 2024 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.