AI Paper Atlas

Research map / Graph learning / GNN architectures

Exphormer: Sparse Transformers for Graphs

Hamed Shirzad, Ameya Velingker, Balaji Venkatachalam, Danica J. Sutherland, Ali Kemal Sinop

ICML 2023 · Graph learning › GNN architectures · approach: Expressive equivariant message passing · 7 citations (OpenAlex)

Read the paper

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
add Exphormer sparse attention (expander + global + local) to GraphGPS instead of full transformerGraphGPS on CIFAR10 · CIFAR10Accuracy: +2.39 (74.69 vs 72.30)Improved
add Exphormer sparse attention instead of full transformerGraphGPS on MalNet-Tiny · MalNet-TinyAccuracy: +0.52 (94.02 vs 93.50)Improved
add Exphormer sparse attention instead of full transformerGraphGPS on Peptides-Func · Peptides-FuncAP: -0.0008 (0.6527 vs 0.6535)Hurt
add Exphormer sparse attention instead of full transformerGraphGPS on PCQM-Contact · PCQM-ContactMRR: +0.03 (0.3637 vs 0.3337)Improved
add Exphormer sparse attention instead of full transformerGraphGPS on ogbn-arxiv (GCN backbone) · ogbn-arxivAccuracy: +1.09 (72.44 vs 71.35 GCN only)Improved
remove expander edges from Exphormer (keep local + global) (ablation)Exphormer on PascalVOC-SP · PascalVOC-SPF1 score: expander edges add +0.0387 (0.3975 vs 0.3588)Improved
remove global virtual nodes from Exphormer (ablation)Exphormer on PascalVOC-SP · PascalVOC-SPF1 score: -0.0293 (0.3682 vs 0.3975 all components, table shows all-components value 0.3682)Mixed
remove expander edges from Exphormer (ablation)Exphormer on MalNet-Tiny · MalNet-TinyAccuracy: 94.02 without expander vs 92.06 with all componentsMixed
remove local neighborhood edges from Exphormer (ablation)Exphormer on CIFAR10, MalNet-Tiny, PATTERN, PascalVOC-SP · multipleaccuracy/F1: CIFAR10 -0.07; PATTERN -0.15; MalNet-Tiny +0.58 (ablation higher); PascalVOC-SP -0.0274Hurt
remove expander edges from Exphormer (ablation)Exphormer on Peptides-Struct · Peptides-StructMAE: 0.2481 without expander vs 0.2643 all components (lower is better, so removing expander is better)Improved
add expander edges to global+local Exphormer (ablation)GCN+Exphormer on ogbn-arxiv · ogbn-arxivAccuracy: +0.22 (72.44 vs 72.22 with virtual nodes only)Improved
use BigBird or Performer sparse attention in GraphGPS on ogbn-arxivGCN backbone · ogbn-arxivAccuracy: -0.19 (BigBird 71.16) and -0.43 (Performer 70.92) vs GCN only 71.35Hurt

Later papers that compared with this method

21 later accepted papers report Exphormer as a baseline in their results tables; 14 of them report a better number on most of the results they share with it.

13 more not listed.

Earlier methods this paper compares with

Named baselines from earlier accepted papers in this paper's results tables: better than 1 of 1 on most shared results.

Matched by method name in results tables read by AI. Authors' own tables tend to favour the newer method, and a comparison can be missed when a method is renamed.

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy