Research map / Graph learning / GNN architectures
Graph Neural Networks Do Not Always Oversmooth
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Uses the Gaussian process equivalence of infinitely wide GCNs to show that, beyond a critical weight variance, GCNs enter a chaotic non-oversmoothing phase where node features stay distinct even at large depth.
Problem
Oversmoothing in deep graph convolutional networks, where node features converge exponentially with depth, prevents designing deep GNNs; the paper asks whether and when this can be avoided.
Approach
Applies information-propagation theory (regular vs. chaotic phases, propagation depth from DNNs) to GCNs via their GP limit, linearizing the multidimensional node-pair covariance dynamics; shows that increasing the initial weight variance beyond a critical value yields a non-oversmoothing phase.
Key results
- On Cora, GCN GPs near the non-oversmoothing transition (sigma_w^2 approx 1) reach accuracy comparable to the originally reported GCN beyond 100 layers.
- On the CSBM, GCN GPs in the chaotic phase remain informative at more than a thousand layers, while oversmoothing networks drop to chance level.
- The critical weight variance for the non-oversmoothing transition is found to be sigma_w^2 approx 1 on Cora and larger than 1 for the CSBM and complete graph examples.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| increase weight variance at initialization past the transition | GCN · complete graph and CSBM | equilibrium feature distance | Helped |
| increase weight variance at initialization into the non-oversmoothing phase | GCN GP · CSBM node classification | generalization error | Helped |
| increase depth while initializing near the non-oversmoothing transition | GCN GP · Cora node classification | accuracy | Helped |
| use weight variance in the oversmoothing phase | GCN GP · CSBM node classification | generalization error | Hurt |
| use weight variance in the chaotic phase rather than the oversmoothing phase | GCN GP · CSBM node classification at more than 1,000 layers | generalization error | Helped |
| use a finite-size GCN and train only its readout layer | GCN · CSBM node classification | performance | Mixed |
| increase weight variance | GCN with the Kipf–Welling shift operator · equilibrium feature distance | average pairwise feature distance | Helped |
Surprising findings
- Increasing the initial weight variance above a critical value prevents oversmoothing, contrary to the belief that GCNs inevitably oversmooth.
- Prior proofs of inevitable oversmoothing rely on weight-norm assumptions that exclude the chaotic non-oversmoothing phase.
Limitations
- Theory strictly holds only in the infinite feature-dimension GP limit; finite-width deviations are expected for small hidden sizes.
- Eigenvalue analysis costs O(N^6) and is expensive for large graphs.
- Analysis mainly uses a row-stochastic shift operator; the standard Kipf-Welling operator is only preliminarily studied.
Experimental setup
Datasets: Contextual Stochastic Block Model (synthetic) Cora
Benchmarks: Contextual Stochastic Block Model Cora
Metrics: node feature distance generalization error classification accuracy
Baselines: GCN
Builds on: Gaussian process equivalence of infinitely wide networks information propagation theory of deep neural networks (Schoenholz et al.) graph convolutional networks
Closest papers by meaning
- Backward Oversmoothing: why is it hard to train deep Graph Neural Networks? (ICML 2026 · similarity 0.93)
- Demystifying Oversmoothing in Attention-Based Graph Neural Networks (NeurIPS 2023 · similarity 0.93)
- A Signed Graph Approach to Understanding and Mitigating Oversmoothing (NeurIPS 2025 · similarity 0.89)
- Deep Hypergraph Neural Networks with Tight Framelets (AAAI 2025 · similarity 0.89)
- A Dynamical Systems-Inspired Pruning Strategy for Addressing Oversmoothing in Graph Attention Networks (ICML 2025 · similarity 0.89)
- On Vanishing Gradients, Over-Smoothing, and Over-Squashing in GNNs: Bridging Recurrent and Graph Learning (NeurIPS 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.