Research map / Graph learning / GNN architectures
Mixture of Scope Experts at Test: Generalizing Deeper Graph Neural Networks with Shallow Variants
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes MoScaT, a decoupled post-hoc attention gating model that mixes independently trained GNN experts of different depths (scopes) at test time, motivated by a PAC-Bayesian subgroup generalization bound showing depth shifts generalization preference across homophily levels.
Problem
Deeper GNNs expand receptive fields useful for heterophilous graphs but suffer degradation and overfitting, and existing deeper-GNN and graph MoE methods yield only marginal gains over shallow variants.
Approach
Train GNN experts of each depth independently, then learn a node-adaptive attention gate on a holdout set (with heterophily-biased sample filtering and scope-aware logit augmentation) that combines expert logits, enforcing hard scoping rather than gradient-based soft scoping.
Key results
- MoSca-T lifts base GNNs across architectures, with average improvements of about 4-12% (e.g., ACMGCN +12.28%, GAT +6.90%, SGC +6.53%) over the best base depth.
- With tuned base GNNs, the method reaches new SOTA on most datasets, e.g., 56.36% on Amazon and 86.72% on Penn94 versus prior best around 54.96% and 84.86%.
- Ablation shows removing the holdout set causes a large drop (e.g., Penn94 84.80 to 77.72 for SGC), and mean-ensembling deep experts does not reliably help.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add node-adaptive expert gating instead of uniform mean ensembling | SGC scope experts · Squirrel, Amazon, Penn, Arxiv | accuracy: 39.65 ± 1.76 -> 43.35 ± 1.97; 50.56 ± 0.53 -> 52.72 ± 0.53; 80.20 ± 0.77 -> 84.80 ± 0.49; 46.68 ± 0.25 -> 51.86 ± 0.30 | Mixed |
| Add a holdout set for gating-model training (ablation) | MOSCAT-SGC · Squirrel | accuracy: 39.86 ± 2.31 -> 43.35 ± 1.97 | Helped |
| Add multiple scope experts instead of using experts with the same best scope (ablation) | MOSCAT-SGC · Arxiv | accuracy: 46.63 ± 0.28 -> 51.86 ± 0.30 | Helped |
| Add heterophily-biased sample filtering (ablation) | MOSCAT-SGC · Amazon | accuracy: 50.87 ± 0.88 -> 52.72 ± 0.53 | Helped |
| Add scope-aware logit augmentation (ablation) | MOSCAT-SGC · Penn | accuracy: 82.79 ± 0.49 -> 84.80 ± 0.49 | Helped |
| Use uniform mean weights rather than node-adaptive gating (ablation) | SGC scope experts · Squirrel | accuracy: 43.35 ± 1.97 -> 39.65 ± 1.76 | Hurt |
Surprising findings
- Mean-ensembling experts across depths does not guarantee accuracy gains due to poor deeper experts.
- Improvements are smaller for decoupled architectures (SGC, GAMLP, 1-2%) than for message-passing models (GCN, ACMGCN, 4-10%) when increasing max depth.
Limitations
- Requires a holdout labeled set, reducing data available to experts unless validation set is used.
- Gains are smaller for decoupled architectures such as SGC and GAMLP.
- Excludes small heterophilous benchmarks due to high variance, limiting evaluation scope.
Experimental setup
Datasets: Squirrel Patents arXiv Flickr Amazon Penn94 Genius ogbn-arxiv
Benchmarks: Squirrel Patents arXiv Flickr Amazon Penn94 Genius ogbn-arxiv
Metrics: test accuracy ROC-AUC (Genius)
Baselines: MLP SGC GCN GAT GCNII ACMGCN H2GCN GPRGCN FSGNN MixHop GraphGPS SGFormer
Builds on: GCN SGC GCNII ACM-GCN GAMLP PAC-Bayes subgroup generalization bound
Closest papers by meaning
- Decoupling the Depth and Scope of Graph Neural Networks (NeurIPS 2021 · similarity 0.91)
- Making Classic GNNs Strong Baselines Across Varying Homophily: A Smoothness–Generalization Perspective (NeurIPS 2025 · similarity 0.90)
- Towards Bridging Generalization and Expressivity of Graph Neural Networks (ICLR 2025 · similarity 0.88)
- Integrating Co-Training with Edge Discrimination to Enhance Graph Neural Networks Under Heterophily (AAAI 2025 · similarity 0.87)
- Can Classic GNNs Be Strong Baselines for Graph-level Tasks? Simple Architectures Meet Excellence (ICML 2025 · similarity 0.87)
- Generalization of Graph Neural Networks Is Robust to Model Mismatch (AAAI 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.