Research map / Graph learning / GNN architectures
Do Transformers Really Perform Badly for Graph Representation?
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace Laplacian PE with spatial (SPD) encoding (ablation) | Graph Transformer · PCQM4M-LSC | valid MAE: 0.1483 -> 0.1427 | Helped |
| add spatial encoding (ablation) | Transformer · PCQM4M-LSC | valid MAE: 0.2276 -> 0.1427 | Helped |
| add degree centrality encoding (ablation) | Graphormer · PCQM4M-LSC | valid MAE: 0.1427 -> 0.1396 | Helped |
| edge encoding as attention bias vs via node features (ablation) | Graphormer · PCQM4M-LSC | valid MAE: 0.1328 -> 0.1304 | Helped |
| add edge encoding via attention bias (ablation) | Graphormer · PCQM4M-LSC | valid MAE: 0.1396 -> 0.1304 | Helped |
| Graphormer vs best GNN (GIN-VN) | graph-level prediction · PCQM4M-LSC | valid MAE: 0.1395 -> 0.1234 | Helped |
| widen GT hidden dim to 768 | GT · PCQM4M-LSC | valid MAE: 0.1400 -> 0.1408 | No clear effect |
| scale up Graphormer (Small to base) | Graphormer · PCQM4M-LSC | valid MAE: 0.1264 -> 0.1234 | Helped |
| pretrain on PCQM4M-LSC and fine-tune (GIN-VN) | GIN-VN · OGBG-MolHIV | AUC | Hurt |
| Graphormer pretrained + FLAG vs best GNN | Graphormer · OGBG-MolPCBA | AP: 29.79 -> 31.39 | Helped |
| set embedding dropout 0.1 instead of 0 (ablation) | Graphormer · PCQM4M-LSC | valid MAE | Hurt |
| reduce FFN inner dim from 4d to d (ablation) | Graphormer · PCQM4M-LSC | valid MAE | No clear effect |
Closest papers by meaning
- Generalizable Insights for Graph Transformers in Theory and Practice (NeurIPS 2025 · similarity 0.90)
- Subgraphormer: Unifying Subgraph GNNs and Graph Transformers via Graph Products (ICML 2024 · similarity 0.90)
- Towards Principled Graph Transformers (NeurIPS 2024 · similarity 0.89)
- Polynormer: Polynomial-Expressive Graph Transformer in Linear Time (ICLR 2024 · similarity 0.89)
- G-Adapter: Towards Structure-Aware Parameter-Efficient Transfer Learning for Graph Transformer Networks (AAAI 2024 · similarity 0.89)
- What Improves the Generalization of Graph Transformers? A Theoretical Dive into the Self-attention and Positional Encoding (ICML 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.