Research map / Graph learning / GNN architectures
GATE: How to Keep Out Intrusive Neighbors
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes GATE, a GAT extension whose attention separates self and neighbor contributions so it can switch off neighborhood aggregation, improving over-smoothing, depth, and heterophilic performance.
Problem
Graph Attention Networks cannot effectively switch off task-irrelevant neighborhood aggregation, which causes over-smoothing, limits benefit from depth, and hurts performance on heterophilic graphs.
Approach
GATE modifies GAT's attention logits to use separate attention vectors for self-loop and neighbor terms (a_s, a_t) and separate transformation matrices (U, V), with ReLU instead of LeakyReLU, so attention can drive neighbor weights toward zero; a modified conservation law from gradient-flow analysis of GAT motivates it.
Key results
- On ogb-arxiv, GATE reaches 79.57 ± 0.84% test accuracy with 12 layers versus 71.87 ± 0.16% for GAT, claimed as a new SOTA over the 78.03% leaderboard result.
- On the synthetic self-sufficient task with random labels on Cora structure (5 layers), GAT reaches 32.1% while GATE reaches 99.9% test accuracy.
- On heterophilic roman-empire, GATE achieves 89.51 ± 0.49% versus GAT 80.87 ± 0.30% in the extensive comparison setting.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use GATE instead of GAT | GAT node classification · roman-empire | accuracy: 26.10 ± 1.25 (5) -> 75.55 ± 0.30 (5) | Helped |
| Use GATE instead of GAT | GAT node classification · amazon-ratings | accuracy: 45.58 ± 0.41 (10) -> 45.73 ± 0.24 (10) | No clear effect |
| Use GATE instead of GAT | GAT node classification · questions | AUC-ROC: 57.72 ± 1.58 (5) -> 62.95 ± 0.71 (5) | Helped |
| Use GATE instead of GAT | GAT node classification · minesweeper | AUC-ROC: 50.83 ± 0.41 (5) -> 66.14 ± 1.57 (5) | Helped |
| Use GATE instead of GAT | GAT node classification · tolokers | AUC-ROC: 63.57 ± 1.03 (10) -> 66.63 ± 1.15 (10) | Helped |
| Use GATE instead of GAT | OGB node classification · OGB-arxiv | accuracy: 71.87 ± 0.16 -> 79.57 ± 0.84 | Helped |
| Use GATE instead of GAT | OGB node classification · OGB-products | accuracy: 80.63 ± 0.46 -> 86.24 ± 1.01 | Helped |
| Use GATE instead of GAT | OGB node classification · OGB-mag | accuracy: 32.61 ± 0.29 -> 35.29 ± .36 | Helped |
| Use GATE-sep instead of GATE | GATE node classification · amazon-ratings | accuracy: 52.49 ± 0.46 -> 54.51 ± 0.38 | Helped |
| Use GATE-sep instead of GATE | GATE node classification · minesweeper | ROC AUC: 92.82 ± 0.90 -> 94.18 ± 0.43 | Helped |
Surprising findings
- Even with zero-initialized attention, GAT cannot switch off neighborhood aggregation, while a single-layer GAT can almost switch it off on random-label tasks.
- GATE with only d additional parameters per layer performs comparably to GAT-sep, which adds d^2 parameters.
Limitations
- Small-scale heterophilic datasets (Texas, Wisconsin, Actor) favor shallow models and may overfit; no skip connections or regularization used in main experiments.
- Insights rely on gradient-flow and conservation-law analysis that assumes infinitesimally small learning rates, approximated in practice.
- Results on the main real-world comparison use a simplified setting without dropout, weight decay, or hyperparameter tuning.
Experimental setup
Datasets: Erdos-Renyi synthetic graphs Cora ogb-arxiv ogb-products ogb-mag roman-empire amazon-ratings questions minesweeper tolokers citeseer actor
Benchmarks: roman-empire amazon-ratings questions minesweeper tolokers ogb-arxiv ogb-products ogb-mag Cora synthetic test bed
Metrics: test accuracy ROC AUC AUROC alpha_vv distribution
Baselines: GAT GAT-sep MLP MLP+GAT FAGCN GCN GraphSAGE GPR-GNN H2GCN GloGNN GT FSGNN
Builds on: GATv2 GAT conservation law (Thm 2.2) FAGCN Xavier initialization
Closest papers by meaning
- HONGAT: Graph Attention Networks in the Presence of High-Order Neighbors (AAAI 2024 · similarity 0.88)
- Graph Attention is Not Always Beneficial: A Theoretical Analysis of Graph Attention Mechanisms via Contextual Stochastic Block Models (ICML 2025 · similarity 0.84)
- Gated Attention for Large Language Models: Non-linearity, Sparsity, and Attention-Sink-Free (NeurIPS 2025 · similarity 0.83)
- Faithful and Accurate Self-Attention Attribution for Message Passing Neural Networks via the Computation Tree Viewpoint (AAAI 2025 · similarity 0.83)
- Faster Neighborhood Attention: Reducing the O(n^2) Cost of Self Attention at the Threadblock Level (NeurIPS 2024 · similarity 0.83)
- NLGT: Neighborhood-based and Label-enhanced Graph Transformer Framework for Node Classification (AAAI 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.