Research map / Graph learning / GNN architectures
A Dynamical Systems-Inspired Pruning Strategy for Addressing Oversmoothing in Graph Attention Networks
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes DYNAMO-GAT, a GAT variant that uses noise-injected covariance analysis and anti-Hebbian gradual edge pruning with weight recalibration to counter oversmoothing in deep GATs, with theory framed as dynamical-systems lemmas and experiments on Cora, Citeseer, Cornell and synthetic Syn_Products.
Problem
Oversmoothing in deep graph neural networks, where node representations homogenize as depth increases, reducing expressiveness and accuracy.
Approach
Frames oversmoothing as convergence to low-dimensional attractors and prunes attention edges using a noise-driven covariance-based anti-Hebbian criterion with a depth-scaled pruning rate, plus renormalization of remaining attention weights.
Key results
- DYNAMO-GAT reports best accuracy on Cora (83.21), Citeseer (82.01) and Cornell (62.56) in the comparison table, slightly below G2GAT on Cora (83.27) and Citeseer (82.06 vs 82.01 is marginal).
- On Cora, DYNAMO-GAT reaches 137.53 accuracy/GFLOPS versus 136.28 for GCN and 35.11 for GAT.
- Oversmoothing coefficient is reported as roughly constant across layers for DYNAMO-GAT, while GCN and GAT decline rapidly with depth (qualitative figure results).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use DYNAMO-GAT instead of G2GAT | GNN node classification · Cora | accuracy: 83.27 -> 83.21 | Hurt |
| Use DYNAMO-GAT instead of G2GAT | GNN node classification · Citeseer | accuracy: 82.06 -> 82.01 | Hurt |
| Use DYNAMO-GAT instead of G2GAT | GNN node classification · Cornell | accuracy: 61.55 -> 62.56 | Helped |
| Use DYNAMO-GAT instead of G2GAT | GNN node classification · Cora | GFLOPS: 1.209 -> 0.605 | Helped |
| Use DYNAMO-GAT instead of G2GAT | GNN node classification · Citeseer | GFLOPS: 2.452 -> 1.675 | Helped |
| Use DYNAMO-GAT instead of G2GAT | GNN node classification · Cornell | GFLOPS: 0.0879 -> 0.051 | Helped |
| Use DYNAMO-GAT instead of G2GAT | GNN node classification · Cora | Accuracy/GFLOPS: 68.88 -> 137.53 | Helped |
| Use DYNAMO-GAT instead of G2GAT | GNN node classification · Citeseer | Accuracy/GFLOPS: 33.47 -> 48.96 | Helped |
| Use DYNAMO-GAT instead of G2GAT | GNN node classification · Cornell | Accuracy/GFLOPS: 700.34 -> 1226.67 | Helped |
| Use DYNAMO-GAT rather than GCN or GAT across increasing depth | Deep GNN node classification · Cora, Citeseer, and Cornell | test accuracy | Helped |
| Use DYNAMO-GAT rather than GCN or GAT across increasing depth | Deep GNN node classification · Cora, Citeseer, and Cornell | oversmoothing coefficient | Helped |
| Use DYNAMO-GAT rather than GCN, GAT, or G2GAT | Syn_Products · Syn_Products across layers | test accuracy | Helped |
Surprising findings
- Reported DYNAMO-GAT Cora accuracy (83.21) is slightly lower than G2GAT (83.27), contradicting the text's claim of consistently highest accuracy.
Limitations
- Theoretical lemmas rely on informal proof sketches and assumptions (e.g., contraction, rank preservation by induction) that are not rigorously justified.
- Pruning-probability formula and the claim that pruning reduces Jacobian spectral radius are asserted without demonstration.
- Baseline set is small (GCN, GAT, G2GAT) and hyperparameters, seeds and training details are largely unspecified.
Experimental setup
Datasets: Cora Citeseer Cornell Syn_Products Syn_Cora
Benchmarks: Cora Citeseer Cornell Syn_Products
Metrics: test accuracy oversmoothing coefficient (mu) GFLOPS accuracy/GFLOPS
Baselines: GCN GAT G2GAT
Builds on: Graph Attention Networks G2GAT Graph Convolutional Networks
Closest papers by meaning
- Graph Neural Networks Do Not Always Oversmooth (NeurIPS 2024 · similarity 0.89)
- On Oversquashing in Graph Neural Networks Through the Lens of Dynamical Systems (AAAI 2025 · similarity 0.87)
- Graph Attention is Not Always Beneficial: A Theoretical Analysis of Graph Attention Mechanisms via Contextual Stochastic Block Models (ICML 2025 · similarity 0.87)
- When Graph Neural Networks Meet Dynamic Mode Decomposition (ICLR 2025 · similarity 0.87)
- On Vanishing Gradients, Over-Smoothing, and Over-Squashing in GNNs: Bridging Recurrent and Graph Learning (NeurIPS 2025 · similarity 0.87)
- A Signed Graph Approach to Understanding and Mitigating Oversmoothing (NeurIPS 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.