Research map / Graph learning / GNN architectures
Towards Scalable and Deep Graph Neural Networks via Noise Masking
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes RMask, a plug-and-play random-walk module with noise masking that replaces the repeated propagation step in model-simplification GNNs (SIGN, S2GC, GBP, GAMLP) to enable deeper propagation and faster pre-processing.
Problem
Model-simplification GNNs suffer from over-smoothing as propagation depth grows, and their pre-processing (repeated feature propagation) dominates end-to-end training time on large graphs.
Approach
Identify redundant (noise) information in each hop using a de-noise matrix built from shortest-path distances, and replace deterministic propagation with biased random walks over Personalized PageRank importance, so each hop's pure information is extracted in parallel and sparsely.
Key results
- SIGN+RMask reaches 72.8±0.3 on ogbn-arxiv vs 71.9±0.1 for SIGN, and 81.1±0.3 vs 78.2±0.3 on ogbn-products.
- S2GC+RMask is more than 2.9x faster than S2GC on all datasets, with up to 4.9x speedup on large-scale data.
- On ogbn-arxiv, GBP+RMask and GAMLP+RMask stay stable or drop slightly as propagation steps grow from 1 to 30, whereas the originals decline significantly.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add RMask to SIGN | SIGN · Cora, Citeseer, Pubmed, ogbn-arxiv, ogbn-products, ogbn-papers100M | accuracy: 82.1\pm0.3 -> 84.3\pm0.6; 72.4\pm0.8 -> 73.6\pm0.6; 79.5\pm0.5 -> 80.3\pm0.7; 71.9\pm0.1 -> 72.8\pm0.3; 78.2\pm0.3 -> 81 | Helped |
| Add RMask to S$^2$GC | S$^2$GC · Cora, Citeseer, Pubmed, ogbn-arxiv, ogbn-products, ogbn-papers100M | accuracy: 82.5\pm0.3 -> 83.8\pm0.5; 73.0\pm0.2 -> 73.8\pm0.5; 79.6\pm0.3 -> 80.2\pm0.5; 71.8\pm0.3 -> 72.7\pm0.3; 77.1\pm0.5 -> 78 | Helped |
| Add RMask to GBP | GBP · Cora, Citeseer, Pubmed, ogbn-arxiv, ogbn-products, ogbn-papers100M | accuracy: 83.5\pm0.7 -> 84.3\pm0.6; 72.6\pm0.5 -> 73.7\pm0.5; 80.6\pm0.4 -> 81.1\pm0.6; 71.4\pm0.2 -> 71.6\pm0.6; 77.3\pm0.3 -> 78 | Helped |
| Add RMask to GAMLP | GAMLP · Cora, Citeseer, Pubmed, ogbn-arxiv, ogbn-products, ogbn-papers100M | accuracy: 82.3\pm0.4 -> 83.6\pm0.4; 72.6\pm0.6 -> 73.3\pm0.5; 79.1\pm0.7 -> 80.3\pm0.5; 71.9\pm0.3 -> 72.9\pm0.2; 80.3\pm0.3 -> 81 | Helped |
| Add RMask to S$^2$GC | S$^2$GC · All datasets | end-to-end training speedup: more than 2.9 times; up to 4.9 times | Helped |
| Remove noise information identification (ablation) | RMask with S$^2$GC · Cora and Pubmed | accuracy: 82.8\pm0.6 -> 82.1\pm0.3; 84.3\pm0.6 -> 82.1\pm0.3; 84.6\pm0.5 -> 82.1\pm0.3; 79.3\pm0.7 -> 79.5\pm0.5; 80.3\pm0.7 -> 79 | Hurt |
| Remove neighbor nodes importance assignment (ablation) | RMask with S$^2$GC · Cora and Pubmed | accuracy: 82.8\pm0.6 -> 82.3\pm0.2; 84.3\pm0.6 -> 83.9\pm0.2; 84.6\pm0.5 -> 84.1\pm0.3; 79.3\pm0.7 -> 78.8\pm0.4; 80.3\pm0.7 -> 79 | Hurt |
| Add DAGNN (ablation) | RMask with S$^2$GC · Cora and Pubmed | accuracy: 82.8\pm0.6 -> 83.3\pm0.6; 84.3\pm0.6 -> 84.7\pm0.4; 84.6\pm0.5 -> 84.9\pm0.2; 79.3\pm0.7 -> 79.9\pm0.8; 80.3\pm0.7 -> 80 | Helped |
Surprising findings
- GBP+RMask is slightly below GBP on ogbn-arxiv (71.6 vs 71.4 is higher, but GAMLP+RMask on ogbn-products is above GAMLP only marginally) and S2GC+RMask underperforms GBP+RMask-level results on several datasets, indicating gains are not uniform across backbones.
Limitations
- Over-smoothing analysis relies on a cosine-similarity graph smoothness metric and limited hop ranges (up to 30 propagation steps).
- Neighbor importance assignment and noise masking add hyperparameters (number of random walks, restart probability) that trade accuracy against runtime.
- Evaluation is limited to node classification on citation and OGB graphs with standard splits.
Experimental setup
Datasets: Cora Citeseer Pubmed ogbn-arxiv ogbn-products ogbn-papers100M
Benchmarks: Cora Citeseer Pubmed ogbn-arxiv ogbn-products ogbn-papers100M
Metrics: test accuracy end-to-end training time speedup graph smoothness level
Baselines: SIGN S2GC GBP GAMLP DAGNN
Builds on: SGC SIGN S2GC GBP GAMLP DAGNN
Closest papers by meaning
- Generative Graph Pattern Machine (NeurIPS 2025 · similarity 0.89)
- Decoupling the Depth and Scope of Graph Neural Networks (NeurIPS 2021 · similarity 0.89)
- A Comprehensive Study on Large-Scale Graph Training: Benchmarking and Rethinking (NeurIPS 2022 · similarity 0.89)
- Can Classic GNNs Be Strong Baselines for Graph-level Tasks? Simple Architectures Meet Excellence (ICML 2025 · similarity 0.88)
- Bonsai: Gradient-free Graph Condensation for Node Classification (ICLR 2025 · similarity 0.88)
- NoisyGL: A Comprehensive Benchmark for Graph Neural Networks under Label Noise (NeurIPS 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.