Research map / Vision-language and multimodal / Human and embodied modeling
Understanding Transfer Learning of RNA Foundation Models on Downstream Tasks
Yuan Li, Heng Yang, Renzhi Chen, Ke Li
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| fine-tune from pretrained weights instead of frozen last-layer probing | RNA FMs · Archive2, SSI | task metric | Improved |
| use frozen pretrained features instead of one-hot encoding | OmniGenome-52M, PlantRNA-FM · Archive2, bpRNA | F1 | Improved |
| use frozen pretrained features instead of one-hot encoding | RNA FMs · ncRNA | accuracy | Hurt |
| probe intermediate layers (L/4, L/2, 3L/4) instead of the final layer | RNA FMs · downstream tasks | task metric | Mixed |
| start from pretrained weights instead of random initialization (PT-FT vs RI-T) (ablation) | RNA FMs · nine downstream tasks | task metric | Improved |
| pretrained init vs random init (ablation) | RNA-BERT (0.5M params) · SSP tasks | F1 | No clear effect |
| pretrained init vs random init (ablation) | mRNA-FM (239M params) · all evaluated tasks | task metric | No clear effect |
| pretrained init vs random init lowers SAM-style sharpness (ablation) | RNA FMs · Archive2, ncRNA, mRNA-ood | SAM-style flatness: lower in all 15 tested model-task pairs | Improved |
| pretrained init vs random init speeds convergence (ablation) | RNA FMs · seven tasks (training loss) | training loss | Improved |
Closest papers by meaning
- Feature Reuse and Scaling: Understanding Transfer Learning with Protein Language Models (ICML 2024 · similarity 0.91)
- BEACON: Benchmark for Comprehensive RNA Tasks and Language Models (NeurIPS 2024 · similarity 0.86)
- What DNA Foundation Models Learn Beyond Sequence Composition (NeurIPS 2026 · similarity 0.86)
- Exploring Molecular Pretraining Model at Scale (NeurIPS 2024 · similarity 0.86)
- Multi-modal Transfer Learning between Biological Foundation Models (NeurIPS 2024 · similarity 0.86)
- C3P: Contrastive promoter-protein pretraining yields representations capturing bacterial gene regulation (NeurIPS 2026 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
