Research map / Architectures and efficiency / Attention and transformers
SWAMamba: A Sliding Window Attention Mamba Framework for Predicting Translation Elongation Rates
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SWAMamba, combining sliding-window attention with bidirectional Mamba to predict per-codon ribosome density (translation elongation rates) from mRNA coding sequences across four species.
Problem
Predicting ribosome elongation rates at each codon requires modeling both distal codon interactions over long CDS sequences and proximal codons around the ribosomal A site, which existing methods trade off between.
Approach
Stacks two sliding-window attention layers around a bidirectional Mamba block, so the model emphasizes codons near the A site before and after capturing long-range CDS context, with linear complexity in sequence length.
Key results
- SWAMamba achieved average Pearson correlations of 0.596 (C. elegans), 0.630 (S. cerevisiae), 0.641 (humans), and 0.640 (E. coli), above RiboMIMO's 0.576, 0.612, 0.607, 0.572 on the first three and highest across all four.
- Removing dual-stage sliding window attention reduced Pearson correlation to 0.593, 0.621, 0.638, 0.626 across the four species.
- Riboformer's reported performance dropped substantially (0.476-0.552) under the authors' revised data-splitting protocol.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use SWAMamba instead of RiboMIMO | Translation elongation rate prediction · C. elegans | Pearson correlation: 0.576 -> 0.596 | Helped |
| use SWAMamba instead of RiboMIMO | Translation elongation rate prediction · S. cerevisiae | Pearson correlation: 0.612 -> 0.630 | Helped |
| use SWAMamba instead of RiboMIMO | Translation elongation rate prediction · Humans | Pearson correlation: 0.607 -> 0.641 | Helped |
| use SWAMamba instead of RiboMIMO | Translation elongation rate prediction · E. coli | Pearson correlation: 0.572 -> 0.640 | Helped |
| add dual-stage sliding window attention (ablation) | SWAMamba · C. elegans, S. cerevisiae, Humans, E. coli | Pearson correlation: 0.593 -> 0.596; 0.621 -> 0.630; 0.638 -> 0.641; 0.626 -> 0.640 | Helped |
| add the first sliding window attention stage (ablation) | SWAMamba · C. elegans, S. cerevisiae, Humans, E. coli | Pearson correlation: 0.589 -> 0.596; 0.612 -> 0.630; 0.638 -> 0.641; 0.629 -> 0.640 | Helped |
| add the second sliding window attention stage (ablation) | SWAMamba · C. elegans, S. cerevisiae, Humans, E. coli | Pearson correlation: 0.591 -> 0.596; 0.627 -> 0.630; 0.641 -> 0.641; 0.639 -> 0.640 | Mixed |
| add nucleotide features (ablation) | SWAMamba · C. elegans, S. cerevisiae, Humans, E. coli | Pearson correlation: 0.590 -> 0.596; 0.622 -> 0.630; 0.630 -> 0.641; 0.632 -> 0.640 | Helped |
| add amino acid features (ablation) | SWAMamba · C. elegans, S. cerevisiae, Humans, E. coli | Pearson correlation: 0.591 -> 0.596; 0.626 -> 0.630; 0.636 -> 0.641; 0.640 -> 0.640 | Mixed |
Surprising findings
- The original Riboformer data-splitting protocol oversimplified the task, and its performance fell well below its originally reported level under the revised split.
- Codons roughly 80 positions away from the A site showed influence on predicted density comparable to codons near it (CIS analysis in one E. coli example).
Limitations
- Evaluated only on four species; authors plan broader species and conditions.
- Baseline Riboformer required authors to change its data split and modify it to full CDS, so comparison depends on this adaptation.
Experimental setup
Datasets: E. coli ribosome profiling (Mohammad et al. 2019) S. cerevisiae ribosome profiling (Stein et al. 2022) C. elegans ribosome profiling (Stein et al. 2022) Human ribosome profiling (Iwasaki et al. 2016)
Benchmarks: E. coli ribosome profiling S. cerevisiae ribosome profiling C. elegans ribosome profiling human ribosome profiling
Metrics: Pearson correlation coefficient
Baselines: IXnos Riboexp RiboMIMO Riboformer
Builds on: Mamba BiMamba (Caduceus) Longformer sliding window attention RiboMIMO Riboformer
Closest papers by meaning
- HELM: Hierarchical Encoding for mRNA Language Modeling (ICLR 2025 · similarity 0.84)
- Training Compute-Optimal Protein Language Models (NeurIPS 2024 · similarity 0.84)
- Equi-mRNA: Protein Translation Equivariant Encoding for mRNA Language Models (NeurIPS 2025 · similarity 0.84)
- MeToken: Uniform Micro-environment Token Boosts Post-Translational Modification Prediction (ICLR 2025 · similarity 0.84)
- Ultrafast classical phylogenetic method beats large protein language models on variant effect prediction (NeurIPS 2024 · similarity 0.84)
- LEMON-ZEST: Evolution-Informed Tokenization for Efficient Protein Language Modeling (NeurIPS 2026 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.