Research map / Architectures and efficiency / Attention and transformers
Geometric Transformer with Interatomic Positional Encoding
Yusong Wang, Shaoning Li, Tong Wang, Bin Shao, Nanning Zheng, Tie-Yan Liu
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| update IPE across layers via residual connection (vs non-updated IPE) (ablation) | Geoformer · QM9 U0 | MAE (meV): 5.63 -> 4.43 | Improved |
| update IPE across layers via residual connection (vs non-updated IPE) (ablation) | Geoformer · QM9 U | MAE (meV): 5.87 -> 4.41 | Improved |
| update IPE across layers via residual connection (vs non-updated IPE) (ablation) | Geoformer · QM9 H | MAE (meV): 6.21 -> 4.39 | Improved |
| update IPE across layers via residual connection (vs non-updated IPE) (ablation) | Geoformer · QM9 G | MAE (meV): 7.24 -> 6.13 | Improved |
| multiply IPE with attention scores instead of adding as attention bias (ablation) | Geoformer · QM9 U0 | MAE (meV): 7.62 -> 4.43 | Improved |
| multiply IPE with attention scores instead of adding as attention bias (ablation) | Geoformer · QM9 U | MAE (meV): 7.66 -> 4.41 | Improved |
| multiply IPE with attention scores instead of adding as attention bias (ablation) | Geoformer · QM9 H | MAE (meV): 7.93 -> 4.39 | Improved |
| multiply IPE with attention scores instead of adding as attention bias (ablation) | Geoformer · QM9 G | MAE (meV): 9.01 -> 6.13 | Improved |
| Geoformer vs Transformer-M (adds directional info beyond distances, no pretraining) | Geoformer · QM9 U0 | MAE (meV): 9.37 -> 4.43 | Improved |
| Geoformer vs Transformer-M | Geoformer · QM9 eps_LUMO | MAE (meV): 16.2 -> 15.4 | Improved |
| Geoformer vs Transformer-M | Geoformer · QM9 eps_HOMO | MAE (meV): 17.5 -> 18.4 | Hurt |
| Geoformer vs Transformer-M | Geoformer · QM9 delta eps | MAE (meV): 27.4 -> 33.8 | Hurt |
Closest papers by meaning
- GeoMFormer: A General Architecture for Geometric Molecular Representation Learning (ICML 2024 · similarity 0.91)
- Equivariant Spherical Transformer for Efficient Molecular Modeling (NeurIPS 2026 · similarity 0.87)
- Beyond Atoms: Enhancing Molecular Pretrained Representations with 3D Space Modeling (ICML 2025 · similarity 0.87)
- NEAT: Neighborhood-Guided, Efficient, Autoregressive Set Transformer for 3D Molecular Generation (NeurIPS 2026 · similarity 0.86)
- E2Former: An Efficient and Equivariant Transformer with Linear-Scaling Tensor Products (NeurIPS 2025 · similarity 0.86)
- Towards Unified and Lossless Latent Space for 3D Molecular Latent Diffusion Modeling (NeurIPS 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
