Research map / Graph learning / GNN architectures
GrokFormer: Graph Fourier Kolmogorov-Arnold Transformers
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes GrokFormer, a graph transformer whose spectral filter is built from learnable Fourier-series activations over K-order Laplacian powers with learnable order coefficients, enabling adaptive spectrum and order.
Problem
Self-attention in graph transformers is low-pass and existing spectral GT filters are fixed in spectral order or spectrum, limiting expressiveness for high-frequency and heterophilic graph signals.
Approach
Graph Fourier KAN: replaces spline-based KAN activations with Fourier series over eigenvalue powers λ^k, producing K filter bases combined by learnable order coefficients α_k, so the filter is adaptive in both spectrum and spectral order; combined additively with efficient self-attention in a Transformer layer.
Key results
- GrokFormer reaches 89.57% on Cora, 94.59% on Texas and 75.58% on Chameleon, the best among the 11 node datasets reported in the table (Penn94 is 83.59%, second to Specformer's 84.32%).
- On filter fitting with six synthetic filters, GrokFormer attains the lowest SSE on all six, e.g. 0.0029 vs Specformer 0.0283 on Low-comb.
- On graph classification GrokFormer scores 99.5 on MUTAG and 94.8 on PTC-MR, outperforming the strongest baseline Specformer on 4 of 5 datasets.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add adaptive Fourier-series Graph Fourier KAN and efficient self-attention | GrokFormer · Node classification: homophilic datasets | accuracy | Helped |
| Add adaptive Fourier-series Graph Fourier KAN and efficient self-attention | GrokFormer · Penn94 | accuracy: 84.32 vs. 83.59 | Hurt |
| Add adaptive Fourier-series Graph Fourier KAN and efficient self-attention | GrokFormer · Graph classification benchmarks | accuracy | Mixed |
| Add GrokFormer rather than Specformer | Graph classification · PTC-MR | accuracy: 82.9 -> 94.8 | Helped |
| Add Graph Fourier KAN to efficient self-attention (ablation) | GrokFormer · Cora | accuracy: 77.42 -> 89.57 | Helped |
| Add Graph Fourier KAN to efficient self-attention (ablation) | GrokFormer · Penn94 | accuracy: 76.29 -> 83.59 | Helped |
| Add Graph Fourier KAN to efficient self-attention (ablation) | GrokFormer · Texas | accuracy: 90.33 -> 94.59 | Helped |
| Add efficient self-attention to Graph Fourier KAN (ablation) | GrokFormer · Penn94 | accuracy: 81.36 -> 83.59 | Helped |
| Add efficient self-attention to Graph Fourier KAN (ablation) | GrokFormer · Texas | accuracy: 93.62 -> 94.59 | Helped |
| Use Graph Fourier KAN alone instead of efficient self-attention alone (ablation) | GrokFormer · Cora | accuracy: 77.42 -> 88.98 | Helped |
| Use Graph Fourier KAN alone instead of efficient self-attention alone (ablation) | GrokFormer · Penn94 | accuracy: 76.29 -> 81.36 | Helped |
| Use Graph Fourier KAN alone instead of efficient self-attention alone (ablation) | GrokFormer · Texas | accuracy: 90.33 -> 93.62 | Helped |
Surprising findings
- Specformer outperforms GrokFormer on Penn94 (84.32 vs 83.59), the only node-level dataset where GrokFormer is not best.
- Graph Fourier KAN alone (without self-attention) performs strongly, but fails to match the full model on Penn94, which the authors attribute to low homophily.
Limitations
- Relies on spectral decomposition (O(N^3) offline), which is costly for large graphs; large-scale experiments use truncated eigendecomposition with 3,000 or 6,000 eigenvectors.
- Graph-level results on ZINC and CIFAR10 slightly underperform GRIT, and Peptides-func with edge features underperforms GraphGPS/GRIT.
- Hyperparameter search over Fourier terms M, order K, and other settings is extensive per dataset.
Experimental setup
Datasets: Cora Citeseer Pubmed Photo WikiCS Physics Penn94 Chameleon Squirrel Actor Texas PROTEINS
Benchmarks: Cora Citeseer Pubmed Photo WikiCS Physics Penn94 Chameleon Squirrel Actor Texas PROTEINS
Metrics: accuracy SSE R^2 MAE AP
Baselines: GCN GAT H2GCN HopGNN ChebyNet GPRGNN BernNet JacobiConv HiGCN Transformer GraphGPS NodeFormer
Builds on: Specformer BernNet JacobiConv Kolmogorov-Arnold Networks
Closest papers by meaning
- Position: Spectral GNNs Rely Less on Graph Fourier Basis than Conceived (ICML 2025 · similarity 0.90)
- SLOG: An Inductive Spectral Graph Neural Network Beyond Polynomial Filter (ICML 2024 · similarity 0.90)
- HoloNets: Spectral Convolutions do extend to Directed Graphs (ICLR 2024 · similarity 0.89)
- Unifying Homophily and Heterophily for Spectral Graph Neural Networks via Triple Filter Ensembles (NeurIPS 2024 · similarity 0.88)
- Polynormer: Polynomial-Expressive Graph Transformer in Linear Time (ICLR 2024 · similarity 0.88)
- How Universal Polynomial Bases Enhance Spectral Graph Neural Networks: Heterophily, Over-smoothing, and Over-squashing (ICML 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.