Research map / Graph learning / Graph algorithms and clustering
From block-Toeplitz matrices to differential equations on graphs: towards a general theory for scalable masked Transformers
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use GKAT instead of GAT | GAT · Pubmed | accuracy: 77.2 -> 78.0 | Helped |
| Use GKAT instead of GAT | GAT · Cora | accuracy: 83.0 -> 82.1 | Hurt |
| Use GKAT instead of GAT | GAT · Citeseer | accuracy: 72.5 -> 73.0 | Helped |
| Use dynamic random walks generated each epoch (ablation) | GKAT · Citeseer | accuracy: 73.0 -> 73.3 | Helped |
| Use two-layer GKAT instead of GAT | GAT · Induced Cycle detection | accuracy: 77.0% -> 83.6% | Helped |
| Use GKAT instead of GAT on 100K-node graphs | GAT · Caveman 100K | accuracy: 75.0% -> 89.3% | Helped |
| Use GKAT instead of GAT for efficiency | GAT · Pubmed | train speedup: 9.50x | Helped |
| Use GKAT instead of GAT for efficiency | GAT · Tree | train speedup: 1.10x | Helped |
| Replace brute-force RWGNK masking (GKAT-0) with low-rank GKAT (ablation) | GKAT-0 · Citeseer/Pubmed/motifs | accuracy loss: 0.05% | No clear effect |
| Replace brute-force masking with GKAT on memory (ablation) | GKAT-0 · Pubmed | memory ratio: 0.07 | Helped |
| Increase random walk length beyond 4 (ablation) | GKAT · Cora, Citeseer, Pubmed | accuracy | No clear effect |
| Apply 2-level block-Toeplitz masking on top of Performer | vision Performer · ImageNet | accuracy: +2.5-3.4 percentage point | Helped |
Closest papers by meaning
- Linear Transformer Topological Masking with Graph Random Features (ICLR 2025 · similarity 0.88)
- On the Role of Attention Masks and LayerNorm in Transformers (NeurIPS 2024 · similarity 0.87)
- Unifying and Enhancing Graph Transformers via a Hierarchical Mask Framework (NeurIPS 2025 · similarity 0.87)
- Graph Convolutions Enrich the Self-Attention in Transformers! (NeurIPS 2024 · similarity 0.87)
- Clustering in Causal Attention Masking (NeurIPS 2024 · similarity 0.87)
- Mind the Gap: a Spectral Analysis of Rank Collapse and Signal Propagation in Attention Layers (ICML 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.