Research map / Vision-language and multimodal / Cross-modal learning
Bimodal masked language modeling for bulk RNA-seq and DNA methylation representation learning
Maxence Gélard, Hakim Benkirane, Thomas Pierrot, Guillaume Richard, Paul-Henry Cournède
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add bimodal masked LM pre-training (ablation) | MOJO · TCGA pan-cancer classification | test weighted-F1: 0.891 -> 0.952 | Improved |
| add pre-training (ablation) | MOJO-Transformer · TCGA pan-cancer classification | test weighted-F1: 0.827 -> 0.942 | Improved |
| use hybrid convolution-transformer instead of pure transformer (ablation) | MOJO · TCGA pan-cancer classification | test weighted-F1: 0.942 -> 0.952 | Improved |
| use hybrid convolution-transformer instead of pure transformer (ablation) | MOJO · classification training | update time per step (s): 17.987 -> 0.059 | Improved |
| fine-tune with IA3 instead of probing only (ablation) | MOJO · TCGA pan-cancer classification | test weighted-F1: 0.945 -> 0.952 | Improved |
| use bimodal MOJO instead of unimodal BulkRNABert | MOJO · TCGA pan-cancer classification | test weighted-F1: 0.943 -> 0.952 | Improved |
| use joint bimodal modeling instead of late integration (concat.) | MOJO · TCGA pan-cancer survival | C-index: 0.756 -> 0.771 | Improved |
| joint modeling instead of late integration | MOJO · zero-shot pan-cancer | accuracy: 0.870 -> 0.928 | Improved |
| joint modeling instead of late integration | MOJO · zero-shot PAM50 | accuracy: 0.763 -> 0.777 | Improved |
| add mutual information auxiliary loss during fine-tuning (ablation) | MOJO · pan-cancer, RNA-seq dropped from all test samples | test weighted-F1: 0.538 -> 0.916 | Improved |
| add mutual information loss when no modality is missing (ablation) | MOJO · pan-cancer, full test set | test weighted-F1: 0.952 -> 0.949 | No clear effect |
| add extended pre-training with missing-modality samples (ablation) | MOJO · pan-cancer missing RNA-seq | test weighted-F1 | Improved |
Closest papers by meaning
- Multi-modal Transfer Learning between Biological Foundation Models (NeurIPS 2024 · similarity 0.85)
- Leverage the Explainability of Transformer Models to Improve the DNA 5-Methylcytosine Identification (Student Abstract) (AAAI 2024 · similarity 0.84)
- BioXArena: Benchmarking LLM Agents on Multi-Modal Biomedical Machine Learning Tasks (NeurIPS 2026 · similarity 0.84)
- JanusDNA: A Powerful Bi-directional Hybrid DNA Foundation Model (NeurIPS 2025 · similarity 0.84)
- BEACON: Benchmark for Comprehensive RNA Tasks and Language Models (NeurIPS 2024 · similarity 0.84)
- What DNA Foundation Models Learn Beyond Sequence Composition (NeurIPS 2026 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
