Research map / Graph learning / Graph self-supervised learning
Triplet Interaction Improves Graph Transformers: Accurate Molecular Graph Learning with Triplet Graph Transformers
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes the Triplet Graph Transformer (TGT), adding third-order triplet attention/aggregation to edge-augmented graph transformers, and a distance-predictor-based three-stage training with stochastic inference for molecular property prediction, reaching SOTA on PCQM4Mv2, OC20 IS2RE, QM9, MOLPCBA, LIT-PCBA and TSP.
Problem
Pure graph transformers only model second-order pairwise interactions, limiting geometric understanding needed for molecular property prediction, and existing geometric methods depend on expensive 3D coordinates.
Approach
Introduces triplet interaction on pairwise edge embeddings, with triplet attention (O(N^3), most expressive) and triplet aggregation (subcubic, vector values, no junction-node dependence), plus a 2D-to-distance predictor trained in three stages and used with stochastic inference over sampled distance matrices.
Key results
- TGT-At achieves 68.6 meV validation MAE on PCQM4Mv2 (67.1 meV with RDKit coordinates), outperforming UniMol+ at 69.3 meV.
- On OC20 IS2RE, TGT-At reaches 414.7 meV test MAE and 8.30% EwT, the best direct method on the leaderboard.
- TGT-Ag+TGT-At-DP reaches 31.67 AP on MOLPCBA, above the prior best pretrained Graphormer at 31.40.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add triplet attention with RDKit coordinates | TGT on PCQM4Mv2 · PCQM4Mv2 test-dev | MAE (meV): 70.5 -> 68.3 | Helped |
| Add triplet attention | TGT on OC20 IS2RE · OC20 IS2RE test average | MAE (meV): 414.3 vs 414.7 | No clear effect |
| Use the pretrained TGT-At distance predictor instead of RDKit distances | MOLPCBA EGT task predictor · MOLPCBA test | AP (%): 31.09±0.33 -> 31.12±0.25 | No clear effect |
| Use the pretrained TGT-At distance predictor instead of RDKit distances | TGT-Ag task predictor · MOLPCBA test | AP (%): 31.44±0.29 -> 31.67±0.31 | Helped |
| Add the pretrained TGT-At distance predictor | EGT task predictor · LIT-PCBA average test | ROC-AUC (%): 81.2 -> 81.5 | Helped |
| Use TGT-Ag instead of EGT | TSP model, ≈500K parameters · TSP test | F1 (%): 85.3± 0.1 -> 85.7± 0.1 | Helped |
| Remove 3rd-order interaction (ablation) | Distance predictor · PCQM4Mv2 validation-3D | Cross-entropy: 1.199 -> 1.270 | Hurt |
| Remove gating from triplet attention (ablation) | Distance predictor · PCQM4Mv2 validation-3D | Cross-entropy: 1.199 -> 1.207 | Hurt |
| Add stochastic inference (ablation) | PCQM4Mv2 model · PCQM4Mv2 validation | MAE (meV): 85.1 -> 84.2 | Helped |
| Add RDKit coordinates as input (ablation) | PCQM4Mv2 model · PCQM4Mv2 validation | MAE (meV): 84.2 -> 82.2 | Helped |
| Add the denoising objective (ablation) | PCQM4Mv2 model · PCQM4Mv2 validation | MAE (meV): 82.2 -> 80.9 | Helped |
| Add locally smooth input noise (ablation) | PCQM4Mv2 model · PCQM4Mv2 validation | MAE (meV): 80.9 -> 80.5 | Helped |
Surprising findings
- Triplet aggregation with subcubic cost outperforms triangular update and axial attention in distance prediction, showing vector-valued aggregation helps.
- Predicting distances from 2D graphs alone (no RDKit coordinates) still outperforms UniMol+, which uses RDKit coordinates.
Limitations
- Triplet interaction complexity is at least O(N^2.37) or O(N^3), higher than EGT's O(N^2).
- Trade-off between performance and compute is not explored.
- Triplet attention on TSP was not run at the 500K parameter budget due to memory constraints.
Experimental setup
Datasets: PCQM4Mv2 OC20 IS2RE QM9 MOLPCBA LIT-PCBA TSP (Dwivedi et al. 2020)
Benchmarks: PCQM4Mv2 OC20 IS2RE QM9 MOLPCBA LIT-PCBA TSP
Metrics: MAE EwT AP ROC-AUC F1
Baselines: UniMol+ EGT GEM-2 Transformer-M GPS++ Graphormer GraphGPS TokenGT GRPE Equiformer
Builds on: Edge-augmented Graph Transformer (EGT) axial attention triangular update Transformer-M GEM-2
Closest papers by meaning
- GraphGPT: Generative Pre-trained Graph Eulerian Transformer (ICML 2025 · similarity 0.89)
- Generalist Equivariant Transformer Towards 3D Molecular Interaction Learning (ICML 2024 · similarity 0.87)
- Graph Generative Pre-trained Transformer (ICML 2025 · similarity 0.87)
- Empowering Active Learning for 3D Molecular Graphs with Geometric Graph Isomorphism (NeurIPS 2024 · similarity 0.87)
- RISE: Radius of Influence based Subgraph Extraction for 3D Molecular Graph Explanation (ICML 2025 · similarity 0.86)
- Efficient Sharpness-Aware Minimization for Molecular Graph Transformer Models (ICLR 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.