Research map / Architectures and efficiency / RNNs and normalization
GeoMFormer: A General Architecture for Geometric Molecular Representation Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes GeoMFormer, a two-stream Transformer with invariant and equivariant streams bridged by cross-attention, achieving strong results on OC20, PCQM4Mv2, Molecule3D, and N-body simulation.
Problem
Learning molecular representations that are simultaneously invariant (scalar properties) and equivariant (vector/position outputs) under SE(3) with a general, scalable architecture rather than heuristic costly modules.
Approach
Two parallel Transformer streams (invariant and equivariant) with self-attention and bidirectional cross-attention between them; invariant-cross-equivariant and equivariant-cross-invariant attention are designed to preserve SE(3) constraints, and many prior models (PaiNN, TorchMD-Net) are shown to be special instantiations.
Key results
- On OC20 IS2RE, GeoMFormer reaches average energy MAE 0.4141 eV and average EwT 8.59%, versus 0.4410 eV and 6.04% for the best baseline Equiformer.
- On OC20 IS2RS, average ADwT is 10.67% versus 8.36% for GemNet-OC (about 27.6% relative improvement).
- On N-body simulation, MSE is 0.0047 versus 0.0071 for EGNN (33.8% reduction); on Molecule3D it reports MAE 0.0252 (random) and 0.1045 (scaffold).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use GeoMFormer instead of Equiformer | OC20 IS2RE · OC20 IS2RE val, Average | Energy MAE (eV): 0.4410 -> 0.4141 | Helped |
| Use GeoMFormer instead of Equiformer | OC20 IS2RE · OC20 IS2RE val, Average | EwT (%): 6.04 -> 8.59; 42.2% relative EwT improvement | Helped |
| Use GeoMFormer instead of GemNet-OC | OC20 IS2RS · OC20 IS2RS validation, Average | ADwT (%): 8.36 -> 10.67; 27.6% relative ADwT improvement | Helped |
| Use GeoMFormer instead of the previous best model | PCQM4Mv2 · PCQM4Mv2 valid | MAE: 6.7% relative MAE reduction | Helped |
| Use GeoMFormer instead of the previous best model | Molecule3D · Molecule3D random split | MAE: 16.3% relative MAE reduction | Helped |
| Use GeoMFormer instead of the previous best model | Molecule3D · Molecule3D scaffold split | MAE: 11.6% relative MAE reduction | Helped |
| Use GeoMFormer instead of EGNN | N-Body System Simulation · N-Body System Simulation | MSE: 33.8% MSE reduction | Helped |
Limitations
- On PCQM4Mv2, GeoMFormer (0.0734 MAE) is behind Uni-Mol+ (0.0708), which uses more complex settings.
- Scalability of model and dataset size is only partially explored, per the authors.
Experimental setup
Datasets: OC20 PCQM4Mv2 Molecule3D N-body simulation dataset MD17
Benchmarks: OC20 IS2RE OC20 IS2RS PCQM4Mv2 Molecule3D N-body simulation MD17
Metrics: MAE EwT ADwT MSE
Baselines: Equiformer GemNet-T DimeNet++ SphereNet Graphormer-3D GNS PaiNN TorchMD-Net Uni-Mol+ GemNet-OC EGNN Tensor Field Network
Builds on: Transformer PaiNN TorchMD-Net Graphormer-3D Uni-Mol+
Closest papers by meaning
- E2Former: An Efficient and Equivariant Transformer with Linear-Scaling Tensor Products (NeurIPS 2025 · similarity 0.89)
- Equivariant Spherical Transformer for Efficient Molecular Modeling (NeurIPS 2026 · similarity 0.88)
- Beyond Atoms: Enhancing Molecular Pretrained Representations with 3D Space Modeling (ICML 2025 · similarity 0.88)
- Generalist Equivariant Transformer Towards 3D Molecular Interaction Learning (ICML 2024 · similarity 0.87)
- Geometric Representation Condition Improves Equivariant Molecule Generation (ICML 2025 · similarity 0.87)
- Swallowing the Bitter Pill: Simplified Scalable Conformer Generation (ICML 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.