Research map / Graph learning / GNN architectures

Graph Neural Networks Need Cluster-Normalize-Activate Modules

NeurIPS 2024 · Graph learning › GNN architectures · approach: GNN generalization theory · 2 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes Cluster-Normalize-Activate (CNA), a plug-and-play GNN update module that k-means clusters node features, normalizes per cluster, and applies a learnable rational activation per cluster to mitigate oversmoothing.

Problem

Oversmoothing in deep message-passing GNNs, where node features converge to a common fixed point, limiting depth and expressivity for node classification, regression, and graph classification.

Approach

Replace the activation in each GNN update step with a three-stage module: k-means clustering of node features into super-nodes, per-cluster nonparametric normalization (instance-norm style), and a per-cluster learnable rational activation that also subsumes the affine transform, so each cluster gets its own projection.

Key results

• CNA reaches 94.18% accuracy on Cora and 95.75% on CiteSeer, outperforming the Papers with Code leaderboard on 8 of 11 node classification datasets.
• GCN+CNA reaches 74.64% on ogbn-arxiv with 389.2k parameters, outperforming baselines such as RevGAT-SelfKD (74.26%, 2.10M parameters).
• On Cora, CNA improves GCN from 81.59% to 93.66% accuracy, and it improves graph classification on Enzymes from 36.97% to 50.00%.

Measured effects reported in the paper

Surprising findings

• Ablation shows that Cluster+Normalize alone gives 93.02% on Cora, while the Normalize step without clustering gives no gain, and that all three steps are needed for the best result.

Limitations

• Evaluated only on popular convolutional MPNN architectures and datasets, not other GNN types.
• Clustering speed may become a bottleneck on larger graphs.
• Does not entirely prevent oversmoothing and was not tested at greater depth or linked formally to oversmoothing theory.

Experimental setup

Datasets: Cora CoraFull CiteSeer PubMed DBLP Computers Photo Chameleon Squirrel Texas Wisconsin ogbn-arxiv

Benchmarks: Cora CoraFull CiteSeer PubMed DBLP Computers Photo Chameleon Squirrel Texas Wisconsin ogbn-arxiv

Metrics: accuracy normalized mean squared error mean squared error Dirichlet energy number of parameters

Baselines: GCN GAT GraphSAGE TransformerConv Dir-GNN PairNorm GCNII G²-GCN DJ-GNN Exphormer SSP RevGAT-SelfKD

Builds on: Message-passing neural networks Rational activation functions GraphNorm Deep Group Normalization PairNorm

Closest papers by meaning

• Do Graph Neural Networks Learn Generalizable Algorithms for Clustering Graphs? (NeurIPS 2026 · similarity 0.88)
• GraphNorm: A Principled Approach to Accelerating Graph Neural Network Training (ICML 2021 · similarity 0.88)
• A Manifold Perspective on the Statistical Generalization of Graph Neural Networks (ICML 2025 · similarity 0.88)
• Can Classic GNNs Be Strong Baselines for Graph-level Tasks? Simple Architectures Meet Excellence (ICML 2025 · similarity 0.87)
• Classic GNNs are Strong Baselines: Reassessing GNNs for Node Classification (NeurIPS 2024 · similarity 0.87)
• Residual Connections and Normalization Can Provably Prevent Oversmoothing in GNNs (ICLR 2025 · similarity 0.87)