Research map / Graph learning / GNN architectures

Mitigating Oversmoothing Through Reverse Process of GNNs for Heterophilic Graphs

ICML 2024 · Graph learning › GNN architectures · approach: Structural GNN variants · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes reverse-process GNNs (ReP) that invert message-passing layers (GRAND via numerical backward integration; GCN/GAT via invertible residual layers with fixed-point inversion) to produce distinguishable representations, mitigating over-smoothing on heterophilic graphs.

Problem

GNN message passing smooths node representations with depth (over-smoothing), which hurts heterophilic node classification where adjacent nodes need different representations and long-range interactions are needed.

Approach

Apply the inverse of forward diffusion-like message-passing layers (g = f^{-1}) to obtain less-smoothed, distinguishable representations; concatenate forward and reverse representations for prediction. Invertibility of residual GCN/GAT layers is enforced via Lipschitz-constrained weight normalization and fixed-point iteration (iResNet-style).

Key results

• GCN+ReP improves over GCN on heterophilic datasets, e.g. Roman-Empire 73.69 to 86.43 and Minesweeper 89.75 to 96.05 ROC-AUC, and reaches state-of-the-art on four of seven heterophilic datasets.
• GAT+ReP improves over GAT on Chameleon (39.21 to 43.24) and Minesweeper (92.01 to 94.89).
• On homophilic datasets ReP changes results only marginally (e.g. GCN+ReP PubMed 89.37 to 89.96), with some decreases such as GCN+ReP Cora 88.00 to 87.63.

Measured effects reported in the paper

Surprising findings

• Reverse depth above 64 layers was consistently used by the best GCN+ReP configurations, and GSL of GCN+ReP stays below 0.6 up to 1024 reverse layers while plain 32/64-layer GCN becomes similar after eight layers.
• Inversion error from fixed-point iteration is negligible (around 1e-5) even at depth 64 with c=0.99999 and only eight iterations.

Limitations

• Invertibility requires Lipschitz constraints on the forward layer and constant hidden dimension, limiting representation power and design choices.
• Reverse process adds time and memory cost that grows with reverse depth and fixed-point iterations; deep reverse depth was memory-constrained on Minesweeper (<=256).

Experimental setup

Datasets: Chameleon Squirrel Roman-Empire Amazon-Ratings Minesweeper Tolokers Questions Cora CiteSeer PubMed Computers Photo

Benchmarks: Chameleon Squirrel Roman-Empire Amazon-Ratings Minesweeper Tolokers Questions Cora CiteSeer PubMed Computers Photo

Metrics: accuracy ROC-AUC graph smoothness level mean absolute inversion error

Baselines: GCN GAT GraphSAGE GT H2GCN GPR-GNN FSGNN GloGNN

Builds on: GRAND iResNet GCN GAT

Closest papers by meaning

• On Vanishing Gradients, Over-Smoothing, and Over-Squashing in GNNs: Bridging Recurrent and Graph Learning (NeurIPS 2025 · similarity 0.88)
• Graph Neural Networks Do Not Always Oversmooth (NeurIPS 2024 · similarity 0.87)
• A Generalized Neural Diffusion Framework on Graphs (AAAI 2024 · similarity 0.87)
• On the Impact of Feature Heterophily on Link Prediction with Graph Neural Networks (NeurIPS 2024 · similarity 0.87)
• Understanding Heterophily for Graph Neural Networks (ICML 2024 · similarity 0.87)
• Deep Graph Neural Networks via Posteriori-Sampling-based Node-Adaptative Residual Module (NeurIPS 2024 · similarity 0.87)