Research map / Graph learning / GNN architectures
KAA: Kolmogorov-Arnold Attention for Enhancing Attentive Graph Neural Networks
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Kolmogorov-Arnold Attention (KAA), which replaces the learnable score mapping in attentive GNN scoring functions with a KAN, and uses a Maximum Ranking Distance metric to show its theoretical expressive-power advantage.
Problem
Scoring functions in attentive GNNs (GAT-based and Transformer-based) are overly simple and limit expressive power and practical performance; existing static/dynamic attention measures are too coarse to compare them.
Approach
Unify attentive GNN scoring functions as an alignment function followed by a learnable score mapping, replace the score mapping with a KAN (B-spline based), and quantify expressiveness via Maximum Ranking Distance, showing a single-layer KAN with zero-order B-splines has near-unlimited ranking capacity with fewer parameters than an MLP.
Key results
- KAA-enhanced models outperform their originals on all node-level datasets, with an average improvement of 1.63% across node tasks.
- On graph-level tasks KAA yields an average 7.71% improvement, with gains exceeding 20% in some cases (e.g., GLCN on PPI, GAT on QM9).
- Theory: MRD bounds show a single-layer KAN with d^2 parameters has near-zero MRD upper bound, while MLP and linear-transformation scoring have MRD lower bounds growing with N^3.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace the original attentive GNN score mapping with KAN | GAT-based attentive GNNs · Node-level tasks (Cora, CiteSeer, PubMed, ogbn-arxiv, Photo, Computers; Cora, CiteSeer, PubMed link prediction) | accuracy or ROC-AUC (%): +1.95; +2.39; +0.82; +1.01; +1.25; +2.73; +2.00; +1.00; +1.47 | Helped |
| Replace the original attentive GNN score mapping with KAN | Transformer-based attentive GNNs GT and SAN · Node-level tasks | accuracy or ROC-AUC (%): +2.70% average performance improvement | Helped |
| Replace the original attentive GNN score mapping with KAN | Attentive GNNs · Graph-level tasks (PPI, MUTAG, ENZYMES, PROTEINS, ZINC, QM9) | accuracy, F1, or MAE: +7.71% average performance enhancement | Mixed |
| Replace GAT score mapping with KAN | GAT · QM9 graph regression | MAE: 0.4749 -> 0.2124 | Helped |
| Replace GLCN score mapping with KAN | GLCN · PPI graph classification | F1: 74.86 ±1.15 -> 94.96 ±0.26 | Helped |
| Replace CFGAT score mapping with KAN | CFGAT · ENZYMES graph classification | accuracy (%): 28.33 ±3.37 -> 37.00 ±3.58 | Helped |
Surprising findings
- Original GAT-based attentive models do not outperform all non-attention models on any node-level dataset, suggesting extra scoring parameters do not help by themselves.
Limitations
- Theoretical MRD analysis relies on simplifying assumptions (circulant alignment matrix, N = d^2, fully connected central node).
- Gains on node-level tasks are modest (around 1-3%), and some results are within standard deviations.
- Hyperparameter search space is large (grid size, spline order, dropout, layers), so comparisons depend on tuning.
Experimental setup
Datasets: Cora CiteSeer PubMed ogbn-arxiv Amazon-Photo Amazon-Computers PPI MUTAG ENZYMES PROTEINS ZINC QM9
Benchmarks: Cora CiteSeer PubMed ogbn-arxiv Amazon-Photo Amazon-Computers PPI MUTAG ENZYMES PROTEINS ZINC QM9
Metrics: accuracy ROC-AUC F1 MAE
Baselines: GCN GraphSAGE GIN GAT GLCN CFGAT GT SAN
Builds on: GAT Graph Transformer Kolmogorov-Arnold Network
Closest papers by meaning
- Towards Deep Attention in Graph Neural Networks: Problems and Remedies (ICML 2023 · similarity 0.87)
- Understanding Attention and Generalization in Graph Neural Networks (NeurIPS 2019 · similarity 0.87)
- A Practical, Progressively-Expressive GNN (NeurIPS 2022 · similarity 0.87)
- Learning to Rank: How GNNs Solve Max-Clique and Sparse PCA (AAAI 2026 · similarity 0.87)
- Beyond Weisfeiler-Lehman: A Quantitative Framework for GNN Expressiveness (ICLR 2024 · similarity 0.85)
- A Manifold Perspective on the Statistical Generalization of Graph Neural Networks (ICML 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.