Research map / Graph learning / GNN architectures
Curriculum-Enhanced Residual Soft An-Isotropic Normalization for Over-Smoothness in Deep GNNs
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes R-SoftGraphAIN, a residual soft graph anisotropic normalization layer, plus SmoothCurriculum, a label-smoothing curriculum on an auxiliary graph, to alleviate over-smoothing and optimization difficulty in deep GNNs.
Problem
Deep GNNs suffer from over-smoothing (over-close node embeddings) and difficult optimization, which degrade node classification performance as depth grows.
Approach
Normalize the covariance of node embeddings anisotropically with a soft (partial-SVD, singular-value-softened) operator combined with residual and initial connections, and train with a curriculum built from iteratively label-smoothed targets on an auxiliary KNN or input graph, moving from coarse to fine node discrimination.
Key results
- Ours(GCN) reaches 85.12% on Cora with 32 layers and 84.87% with 64 layers, versus 85.34% for GCNII at 64 layers as the top baseline on Cora.
- Ours(GCN) gets 85.21% on Amazon Computers at 32 layers, exceeding DAGNN by more than 6.8 points as stated in the paper.
- On heterophilous Texas with 32 layers, Ours(GCN) reaches 85.41% versus 69.19% for GCNII (best baseline in that column).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add R-SoftGraphAIN (ablation) | Ours(GCN) · Cora, 32 layers | accuracy: 40.22±5.71 -> 85.12±0.15 | Helped |
| add SoftGraphAIN (ablation) | Ours(GCN) · Cora, 32 layers | accuracy: 83.30±0.12 -> 85.12±0.15 | Helped |
| add residual connections (ablation) | Ours(GCN) · Cora, 32 layers | accuracy: 82.88±0.79 -> 85.12±0.15 | Helped |
| add label smoothing (ablation) | Ours(GCN) · Cora, 32 layers | accuracy: 83.96±0.65 -> 85.12±0.15 | Helped |
| add curriculum learning (ablation) | Ours(GCN) · Cora, 32 layers | accuracy: 82.40±0.65 -> 85.12±0.15 | Helped |
| apply Ours(GCN) instead of DAGNN | Computers, 32 layers · Computers | accuracy: +6.8% | Helped |
Surprising findings
- Many baselines, including GAT, GIN and BernNet, collapse to near-random accuracy at 64 layers on several datasets, indicating severe degradation in deep settings.
Limitations
- Ablation and hyperparameter studies are limited to a few benchmarks in the main text; many results are deferred to the supplement.
- Experiments use only basic GNN backbones (GCN, GAT, GIN); the authors note more sophisticated backbones are untested.
- Curriculum relies on a pre-trained teacher model and an auxiliary graph choice that varies by dataset type.
Experimental setup
Datasets: Cora Citeseer PubMed Coauthor CS Coauthor Physics Amazon Computers Amazon Photo Texas Wisconsin Cornell Actor OGBN-ArXiv
Benchmarks: Cora Citeseer PubMed Coauthor CS Coauthor Physics Amazon Computers Amazon Photo Texas Wisconsin Cornell Actor OGBN-ArXiv
Metrics: accuracy
Baselines: GCN SGC ChebNet GAT GIN PairNorm GCNII JKNet GPRGNN DAGNN APPNP BernNet
Builds on: GCN SGC PairNorm GCNII Label Propagation
Closest papers by meaning
- Residual Connections and Normalization Can Provably Prevent Oversmoothing in GNNs (ICLR 2025 · similarity 0.90)
- Towards Deeper Graph Neural Networks with Differentiable Group Normalization (NeurIPS 2020 · similarity 0.89)
- Deep Graph Neural Networks via Posteriori-Sampling-based Node-Adaptative Residual Module (NeurIPS 2024 · similarity 0.89)
- Normalize Then Propagate: Efficient Homophilous Regularization for Few-Shot Semi-Supervised Node Classification (AAAI 2025 · similarity 0.86)
- Training Deep Neural Networks with Virtual Smoothing Classes (AAAI 2025 · similarity 0.86)
- On Which Nodes Does GCN Fail? Enhancing GCN From the Node Perspective (ICML 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.