Research map / Label-efficient and robust learning / Recognition and long-tail
GENEB: Why Genomic Models Are Hard to Compare
Daria Ledneva, Mikhail Nuridinov, Denis Kuznetsov
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add multi-species instead of human-only pretraining (6 matched pairs) | Transformer genomic models · GENEB macro-MCC | macro-MCC: +0.018 macro-MCC (5/6 pairs) | Improved |
| multi-species vs human pretraining on chromatin accessibility | matched pairs · GENEB chromatin accessibility | macro-MCC: +0.076 (6/6 pairs) | Improved |
| multi-species vs human pretraining on virus/phage | matched pairs · GENEB virus/phage | macro-MCC: -0.025 | Hurt |
| multi-species eukaryotic vs microbial-only pretraining | NT-v2-100M-MS, Genomics-FM vs DNABERT-S · GENEB macro-MCC | macro-MCC: +0.084 macro-MCC | Improved |
| curated eukaryotic genes vs broad multi-species pretraining | GENERator-Eukaryote-3B vs DNA-GPT-3B-M · GENEB macro-MCC | macro-MCC: +0.063 macro-MCC | Improved |
| Transformer-decoder vs Mamba, matched corpus and tokenization | GenomeOcean-500M vs eccDNAMamba · GENEB macro-MCC | macro-MCC: 0.419 -> 0.550 | Improved |
| Transformer-encoder vs decoder, matched multi-species/BPE | GENA-LM-Large-T2T vs OmniNA-220M · GENEB macro-MCC | macro-MCC: 0.425 -> 0.552 | Improved |
| Mamba vs Transformer-decoder on chromatin accessibility | eccDNAMamba vs GenomeOcean-500M · GENEB chromatin accessibility | macro-MCC: 0.475 -> 0.599 | Improved |
| BPE vs k-mer tokenization, Transformer-encoders multi-species | matched pairs · GENEB macro-MCC | macro-MCC: +0.006 across 5 pairs | No clear effect |
| BPE vs k-mer tokenization, Transformer-decoder | GenomeOcean-4B vs DNA-GPT-3B-M · GENEB macro-MCC | macro-MCC: +0.032 macro-MCC | Improved |
| single-nucleotide vs BPE tokenization, human pretraining | MutBERT · GENEB macro-MCC | macro-MCC: +0.033 over GENA-LM, +0.038 over GROVER | Improved |
| scale up model size (log params) | 40 genomic models · GENEB macro-MCC | Spearman rho: rho 0.573; 0.694 excluding Evo-1-131k | Improved |
Closest papers by meaning
- Fast and Low-Cost Genomic Foundation Models via Outlier Removal (ICML 2025 · similarity 0.86)
- DNABERT-2: Efficient Foundation Model and Benchmark For Multi-Species Genomes (ICLR 2024 · similarity 0.86)
- What DNA Foundation Models Learn Beyond Sequence Composition (NeurIPS 2026 · similarity 0.85)
- Specialized Foundation Models Struggle to Beat Supervised Baselines (ICLR 2025 · similarity 0.85)
- CGBench: Benchmarking Language Model Scientific Reasoning for Clinical Genetics Research (NeurIPS 2025 · similarity 0.85)
- Training Compute-Optimal Protein Language Models (NeurIPS 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
