Research map / Label-efficient and robust learning / Recognition and long-tail
SPACE: Your Genomic Profile Predictor is a Powerful DNA Foundation Model
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SPACE, a supervised genomic-profile-prediction DNA foundation model with species-aware sparse MoE encoder and profile-grouped MoE decoder, reporting SOTA on many NT, GUE and Genomic Benchmarks tasks versus Enformer and NT baselines.
Problem
Pure-sequence unsupervised DNA foundation models lack regulatory context, and existing genomic profile prediction models use shared encoders and independent prediction heads, limiting cross-species and cross-profile representation learning.
Approach
Uses supervised genomic profile prediction as the pre-training objective and adds two MoE modules: a species-aware sparse MoE encoder with species embeddings and a mutual-information loss, and a profile-grouped decoder with dual-gated (group- and expert-level) expert aggregation over profile types.
Key results
- SPACE achieves SOTA on 11 of 18 Nucleotide Transformer downstream tasks, e.g. enhancers MCC 0.631 vs 0.614 for Enformer and splice donors 0.942 vs NT-Multispecies 0.958 (NT-Multispecies wins that task).
- On GUE epigenetic mark prediction SPACE improves over Enformer by large margins, e.g. H3K4me3 49.47 vs 22.19 and COVID classification F1 70.26 vs 61.33.
- On Genomic Benchmarks Drosophila enhancers SPACE reaches 0.721 vs 0.613 for Enformer and 0.586 for CNN.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use SPACE instead of Enformer | Enformer · NT downstream tasks: H2AFZ | MCC: 0.522 ± 0.019 -> 0.548 ± 0.005 | Helped |
| Use SPACE instead of Enformer | Enformer · Genomic Benchmarks: Mouse enhancers | score: 0.835 ± 0.012 -> 0.905 ± 0.010 | Helped |
| Use SPACE instead of Enformer | Enformer · GUE EMP: H3K4me2 | MCC: 29.65 -> 43.80 | Helped |
| Use SPACE instead of the strongest listed baseline | GUE CVC · GUE CVC: Covid | F1-score: 73.04 -> 70.26 | Hurt |
| Use SPACE instead of DNABERT | BEND · Chromatin accessibility | score: 0.85 -> 0.89 | Helped |
| Add the prediction-enhanced decoder (ablation) | SPACE · NT downstream tasks: Splicing | MCC: 0.8977 -> 0.9077 | Helped |
| Add the prediction-enhanced decoder (ablation) | SPACE · NT downstream tasks: Regulatory | MCC: 0.7054 -> 0.7024 | Hurt |
| Add the prediction-enhanced decoder (ablation) | SPACE · GUE EMP | MCC: 0.5339 -> 0.5368 | Helped |
| Replace the enhancement decoder with a parameter-matched MLP (ablation) | SPACE · GUE EMP | MCC: 0.5368 -> 0.5153 | Hurt |
| Replace encoder MoE layers with standard FFNs (ablation) | SPACE · NT downstream tasks: Splicing | MCC: 0.8887 -> 0.9077 | Hurt |
| Remove species embeddings while using standard FFNs in the encoder (ablation) | SPACE · NT downstream tasks: Regulatory | MCC: 0.6986 -> 0.7024 | Hurt |
Surprising findings
- SPACE underperforms on the TATA promoter subset in the NT benchmark, which the authors attribute to that task focusing on simple sequence motifs.
- NT-Multispecies (2.5B) outperforms SPACE on several splicing and TATA tasks despite SPACE being claimed as SOTA on many tasks.
Limitations
- Trained on only two species (human and mouse).
- Model scale (588M sparse) is much smaller than the 2.5B NT variant.
- Does not beat all baselines on every task (e.g., NT-Multispecies on splicing and TATA tasks; Enformer on some GUE human tasks).
Experimental setup
Datasets: Enformer pre-training dataset (human and mouse genomic profiles) Nucleotide Transformer downstream tasks GUE benchmark Genomic Benchmarks BEND
Benchmarks: Nucleotide Transformer downstream tasks GUE Genomic Benchmarks BEND
Metrics: MCC F1-score accuracy Pearson correlation
Baselines: Enformer DNABERT-2 HyenaDNA Nucleotide Transformer Caduceus DNABERT GROVER
Builds on: Enformer Mod-Squad Nucleotide Transformer benchmark
Closest papers by meaning
- What DNA Foundation Models Learn Beyond Sequence Composition (NeurIPS 2026 · similarity 0.86)
- BEND: Benchmarking DNA Language Models on Biologically Meaningful Tasks (ICLR 2024 · similarity 0.84)
- JanusDNA: A Powerful Bi-directional Hybrid DNA Foundation Model (NeurIPS 2025 · similarity 0.83)
- DNABERT-2: Efficient Foundation Model and Benchmark For Multi-Species Genomes (ICLR 2024 · similarity 0.83)
- Hyperbolic Genome Embeddings (ICLR 2025 · similarity 0.83)
- Augmenting Biological Fitness Prediction Benchmarks with Landscapes Features from GraphFLA (NeurIPS 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.