Research map / Probabilistic and scientific modeling / Causal discovery

Learning to Discover Regulatory Elements for Gene Expression Prediction

ICLR 2025 · Probabilistic and scientific modeling › Causal discovery · approach: Differentiable and score-based DAG learning

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Seq2Exp uses a Caduceus-based generator that learns Beta-distributed soft/hard masks over DNA combining sequence and epigenomic signals via an information bottleneck, then predicts CAGE gene expression from the masked sequence, outperforming Enformer, Caduceus, and EPInformer and MACS3 peak calling.

Problem

Predicting gene expression (CAGE) from long DNA sequences and epigenomic signals requires discovering the sparse, long-range regulatory elements that actually drive expression.

Approach

Decompose the mask posterior into a DNA-sequence term and an epigenomic-signal term, combine them as a product of Beta distributions (which stays Beta), and apply an information-bottleneck KL to a Beta prior to enforce sparsity, so the generator selects causal regulatory bases before a predictor regresses expression.

Key results

• Seq2Exp-soft reaches MSE 0.1856 and Pearson 0.8723 on K562 CAGE, versus 0.2197 MSE / 0.8475 Pearson for Caduceus and 0.2140 MSE for EPInformer.
• On GM12878, Seq2Exp-soft achieves MSE 0.1873 and Pearson 0.8951, versus 0.1942 MSE for Caduceus w/ signals.
• Seq2Exp-hard with a ~6.9% mask beats MACS3 peak calling (13.6% mask) on K562 (MSE 0.1863 vs 0.2195).

Measured effects reported in the paper

Surprising findings

• Seq2Exp-hard selects only about 6-7% of bases yet outperforms MACS3 peaks covering 13-16% of the sequence.

Limitations

• Evaluation limited to two cell types (K562, GM12878) and a few epigenomic signals.
• Gains over Caduceus w/ signals are modest (about 0.01 Pearson on K562), with overlapping standard deviations in some cases.
• Core Beta-product assumption (conditional independence of sequence and signal given mask) is a strong simplifying assumption.

Experimental setup

Datasets: ENCODE DNase-seq ENCODE H3K27ac FANTOM5 CAGE 4D Nucleome Hi-C HG38 reference genome

Benchmarks: CAGE gene expression prediction K562 CAGE gene expression prediction GM12878

Metrics: MSE MAE Pearson correlation mask ratio

Baselines: Enformer HyenaDNA Mamba Caduceus Caduceus w/ signals EPInformer MACS3

Builds on: Caduceus information bottleneck EPInformer Gumbel-Softmax

Closest papers by meaning

• What DNA Foundation Models Learn Beyond Sequence Composition (NeurIPS 2026 · similarity 0.85)
• Regulatory DNA Sequence Design with Reinforcement Learning (ICLR 2025 · similarity 0.84)
• STRAND: Sequence-Conditioned Transport for Single-Cell Perturbations (NeurIPS 2026 · similarity 0.83)
• C3P: Contrastive promoter-protein pretraining yields representations capturing bacterial gene regulation (NeurIPS 2026 · similarity 0.83)
• DART-Eval: A Comprehensive DNA Language Model Evaluation Benchmark on Regulatory DNA (NeurIPS 2024 · similarity 0.83)
• Gene Regulatory Network Inference in the Presence of Selection Bias and Latent Confounders (NeurIPS 2025 · similarity 0.83)