Research map / Probabilistic and scientific modeling / Molecules and proteins
Bridging Sequence-Structure Alignment in RNA Foundation Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces OmniGenome, an RNA foundation model pre-trained with single-nucleotide tokenization and structure-contextualised objectives (Str2Seq, Seq2Str, MRLM) on plant transcriptomes, aiming to align RNA sequences and secondary structures bidirectionally.
Problem
RNA foundation models do not establish alignment between sequences and secondary structures, which limits bidirectional sequence-structure mapping for tasks like RNA design and structure prediction.
Approach
Pre-train a single-nucleotide Transformer with a structure-contextualised masked reconstruction objective (Str2Seq), a token-level structure prediction objective (Seq2Str, with ViennaRNA labels), and masked RNA language modelling, so the same model supports both sequence-to-structure and structure-to-sequence mapping.
Key results
- On EternaV2 RNA design, OmniGenome-186M+ solved 74 of 100 puzzles versus 2-3 for RNA-FM, RNA-MSM and SpliceBERT.
- Zero-shot SSP on Archive2 reaches 74.72 F1 for OmniGenome-186M+ versus 73.99 for ViennaRNA.
- OmniGenome-52M reported the best result on several PGB tasks, such as polyadenylation (85.47 F1) and splice site (97.40 F1), over the baselines, per the paper.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add structure-contextualized sequence reconstruction (Str2Seq) | OmniGenome · EternaV2 | accuracy (puzzles solved): 3 -> 74 | Helped |
| Add ViennaRNA structure context (ablation) | OmniGenome-base · Archive2 | F1: 74.38 -> 74.72 | Helped |
| Add ViennaRNA structure context (ablation) | OmniGenome-base · bpRNA | F1: 66.19 -> 66.37 | Helped |
| Add ViennaRNA structure context (ablation) | OmniGenome-base · RNAStralign | F1: 74.91 -> 75.80 | Helped |
| Add structure context (ablation) | OmniGenome-base · mRNA | RMSE: 0.7164 -> 0.7121 | Helped |
| Add structure context (ablation) | OmniGenome-base · SNMD | AUC: 63.81 -> 64.13 | Helped |
| Add structure context (ablation) | OmniGenome-base · SNMR | F1: 50.80 -> 52.44 | Helped |
| Add structure context (ablation) | OmniGenome-base · Archive2, RNAStralign, bpRNA | F1: 90.32 -> 91.89; 97.82 -> 98.21; 83.09 -> 83.18 | Helped |
| Add structure context (ablation) | OmniGenome-base · PGB | F1: 85.47 -> 87.05; 86.87 -> 87.55; 72.45 -> 79.77 | Mixed |
| Use random or noisy structure contexts (ablation) | OmniGenome · zero-shot SSP | F1 | No clear effect |
Surprising findings
- Mamba- and Hyena-style architectures were reported as weak on RNA genome understanding, which the authors attribute to short RNA lengths.
- Baseline RNA FMs (RNA-FM, RNA-MSM) solved only 2-3 EternaV2 puzzles, below the RNAInverse baseline's 30.
Limitations
- Baselines like Uni-RNA and 5UTR-LM were not compared because their code was unavailable or hard to use.
- Benchmark splits were re-filtered with CD-HIT-EST and BLAST, so results are not directly comparable to prior SSP reports; PGB was randomly subsampled to 10k examples.
- Results for OmniGenome+ on PGB were excluded for the largest model due to time-intensive experiments; the paper does not report significance tests.
Experimental setup
Datasets: OneKP bpRNA ArchiveII RNAStralign Rfam Eterna V2 mRNA degradation (Kaggle COVID-19 vaccine) Plant Genomic Benchmark GB GUE
Benchmarks: EternaV2 RGB PGB bpRNA ArchiveII RNAStralign GB GUE
Metrics: accuracy (puzzles solved) macro-F1 F1 AUC RMSE
Baselines: ViennaRNA RNAInverse RNA-FM RNA-MSM SpliceBERT DNABERT-2 Nucleotide Transformer V2 Agro-NT MXFold2 UFold
Builds on: DNABERT-2 ViennaRNA RNA-FM rotary position embeddings
Closest papers by meaning
- BAnG: Bidirectional Anchored Generation for Conditional RNA Design (ICML 2025 · similarity 0.86)
- HELM: Hierarchical Encoding for mRNA Language Modeling (ICLR 2025 · similarity 0.85)
- gRNAde: Geometric Deep Learning for 3D RNA inverse design (ICLR 2025 · similarity 0.85)
- Beyond Sequence: Impact of Geometric Context for RNA Property Prediction (ICLR 2025 · similarity 0.85)
- DNABERT-2: Efficient Foundation Model and Benchmark For Multi-Species Genomes (ICLR 2024 · similarity 0.85)
- BEACON: Benchmark for Comprehensive RNA Tasks and Language Models (NeurIPS 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.