Research map / Vision-language and multimodal / Cross-modal learning
Multimodal Molecular Pretraining via Modality Blending
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes MoleBlend, a relation-level "blend-then-predict" self-supervised method that randomly mixes 2D shortest-path/edge and 3D distance matrices into one attention bias and recovers each modality's relations with a single Transformer.
Problem
Existing multimodal molecular pretraining aligns 2D graphs and 3D conformations only coarsely at the molecule level, using separate encoders per modality, and misses fine-grained atom-relation correspondence.
Approach
Blend three atom-relation matrices (shortest path distance, edge type, 3D Euclidean distance) into one randomly sampled relation matrix fed as attention bias into a single Transformer, then predict all three relation types via an outer-product projection head; theoretically framed as a mutual-information decomposition unifying contrastive, generative and mask-then-predict objectives.
Key results
- Average ROC-AUC of 76.16 across 8 MoleculeNet classification tasks, versus 75.07 for MoleculeSDE (the strongest prior multimodal baseline).
- State-of-the-art on 8 of 12 QM9 tasks among multimodal methods, e.g. Gap 34.75 vs 41.77 MAE for MoleculeSDE.
- Blending ablation: blending outperforms SPD-only, Edge-only and 3D-only masking (e.g. BBBP 71.68 vs 69.02 best mask).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Blend SPD, edge, and 3D relations rather than mask one modality (ablation) | MoleBlend pretraining · MoleculeNet BBBP, BACE, Tox21, and ToxCast | ROC-AUC: BBBP: 68.95, 69.02, 67.60 -> 71.68; BACE: 80.64, 81.97, 80.35 -> 83.41; Tox21: 75.59, 76.01, 75.65 -> 76.58; ToxCast: 62 | Helped |
| Add noisy-node regularization to blend-then-predict (ablation) | MoleBlend pretraining · MoleculeNet BBBP and Tox21; QM9 U298 and U0 | ROC-AUC / MAE: BBBP: 71.59 -> 73.00; Tox21: 75.61 -> 77.82; U298: 14.56 -> 12.02; U0: 15.35 -> 11.82 | Mixed |
| Use blend-then-predict instead of noisy-node alone (ablation) | MoleculeNet 2D pretraining · MoleculeNet classification tasks | ROC-AUC: blend-then-predict outperforms noisy-node on 5 out of 8 tasks | Helped |
| Use blend-then-predict instead of noisy-node alone (ablation) | QM9 3D pretraining · QM9 U298 and U0 | MAE: U298: 14.31 -> 14.56; U0: 13.80 -> 15.35 | Hurt |
| Add 2D molecular information to 3D finetuning (ablation) | MoleBlend · QM9 Alpha, HOMO, and Mu | MAE: Alpha: 0.066 -> 0.060; HOMO: 23.62 -> 21.47; Mu: 0.042 -> 0.037 | Helped |
| Add 3D information to 2D finetuning (ablation) | MoleBlend · MoleculeNet 2D tasks | ROC-AUC / RMSE: BBBP: 73.0 -> 71.8; Tox21: 77.8 -> 76.8; ToxCast: 66.1 -> 67.4; ClinTox: 87.6 -> 90.9; Bace: 83.7 -> 84.3; ESOL: 0.831 - | Mixed |
| Vary the SPD:Edge:3D blending ratio (ablation) | MoleBlend pretraining · MoleculeNet BBBP, BACE, Tox21, ToxCast, and Lipo | ROC-AUC / RMSE: 4:4:2, 3:3:4, 2:2:6, 1:1:8 results: BBBP 72.25, 72.34, 72.52, 72.45; BACE 82.17, 82.47, 82.89, 82.43; Tox21 76.23, 77.19 | No clear effect |
Surprising findings
- Pure 3D Noisy-Node converges much faster on QM9 3D tasks early in training, while MoleBlend overtakes it later.
- Adding 2D information at finetuning on 3D tasks improves results (e.g. HOMO 23.62 to 21.47).
Limitations
- Gains on some tasks are small or absent; MoleBlend underperforms on some 3D tasks (U298, U0) relative to Noisy-Node.
- Several baseline results (e.g., 3D Infomax and GraphMVP under Transformer backbone) are reimplementations, and reported improvements are within reported standard deviations on some tasks.
- Blending ratio and 2D-plus-3D finetuning mainly show marginal differences in ablations.
Experimental setup
Datasets: PCQM4Mv2 MoleculeNet QM9
Benchmarks: MoleculeNet QM9
Metrics: ROC-AUC MAE RMSE
Baselines: AttrMask ContextPred InfoGraph MolCLR GraphCL Mole-BERT GraphMAE GraphMVP 3D InfoMax MoleculeSDE GROVER Distance Prediction
Builds on: GraphMVP 3D Infomax Graphormer noisy node masked auto-encoding
Closest papers by meaning
- Atomas: Hierarchical Adaptive Alignment on Molecule-Text for Unified Molecule Understanding and Generation (ICLR 2025 · similarity 0.88)
- TRIDENT: Tri-Modal Molecular Representation Learning with Taxonomic Annotations and Local Correspondence (NeurIPS 2025 · similarity 0.88)
- Beyond Atoms: Enhancing Molecular Pretrained Representations with 3D Space Modeling (ICML 2025 · similarity 0.88)
- 3D Denoisers Are Good 2D Teachers: Molecular Pretraining via Denoising and Cross-Modal Distillation (AAAI 2025 · similarity 0.88)
- UniCorn: A Unified Contrastive Learning Approach for Multi-view Molecular Representation Learning (ICML 2024 · similarity 0.87)
- 3D-MolT5: Leveraging Discrete Structural Information for Molecule-Text Modeling (ICLR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.