Research map / Graph learning / GNN architectures
Graph Inductive Biases in Transformers without Message Passing
Liheng Ma, Chen Lin, Derek Lim, Adriana Romero-Soriano, Puneet K. Dokania, Mark Coates, Philip Torr, Ser-Nam Lim
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add degree scaler (ablation) | GRIT · ZINC | MAE: 0.076 -> 0.059 | Improved |
| add learned update of RRWP in attention (ablation) | GRIT · ZINC | MAE: 0.066 -> 0.059 | Improved |
| use global attention instead of sparse attention (ablation) | GRIT · ZINC | MAE: 0.066 -> 0.059 | Improved |
| use degree scaler instead of Graphormer degree encoding (ablation) | GRIT · ZINC | MAE: 0.072 -> 0.059 | Improved |
| use GRIT attention instead of Graphormer attention (ablation) | GRIT · ZINC | MAE: 0.117 -> 0.059 | Improved |
| use RRWP instead of RWSE (ablation) | GRIT · ZINC | MAE: 0.081 -> 0.059 | Improved |
| use RRWP instead of SPDPE (ablation) | GRIT · ZINC | MAE: 0.067 -> 0.059 | Improved |
| increase RRWP order K from 2 to 21 (ablation) | GRIT · ZINC | MAE: 0.147 -> 0.059 | Improved |
| vary RRWP order K among 7-42 (ablation) | GRIT · ZINC | MAE: 0.063 (K=7) vs 0.059 (K=21) | No clear effect |
| GRIT attention with RRWP vs other attention modules on k-hop attention learning | single-layer attention · K-hop synthetic 1-hop | MAE: .043 -> .001 | Improved |
| GRIT vs GPS on small datasets | Graph Transformer · CIFAR10 | accuracy: 72.298 -> 76.468 | Improved |
Closest papers by meaning
- Recipe for a General, Powerful, Scalable Graph Transformer (NeurIPS 2022 · similarity 0.90)
- Generalizable Insights for Graph Transformers in Theory and Practice (NeurIPS 2025 · similarity 0.89)
- From Theory to Practice: Rethinking Green and Martin Kernels for Unleashing Graph Transformers (ICML 2025 · similarity 0.88)
- Less is More: on the Over-Globalizing Problem in Graph Transformers (ICML 2024 · similarity 0.88)
- What Improves the Generalization of Graph Transformers? A Theoretical Dive into the Self-attention and Positional Encoding (ICML 2024 · similarity 0.88)
- No Metric to Rule Them All: Toward Principled Evaluations of Graph-Learning Datasets (ICML 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
