Research map / Label-efficient and robust learning / Recognition and long-tail
GraphMETRO: Mitigating Complex Graph Distribution Shifts via Mixture of Aligned Experts
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes GraphMETRO, a mixture-of-aligned-experts GNN that decomposes graph distribution shifts into stochastic transform components and aligns expert representations to a reference model for invariance.
Problem
GNNs generalize poorly under complex, heterogeneous, natural distribution shifts in graphs, which standard invariant learning and single-dimension augmentation methods do not cover.
Approach
A gating GNN predicts instance-specific mixtures over K shift components (graph transformations); K expert encoders each learn representations invariant to one component, aligned to a reference model's representation space via a distance penalty, and aggregated by gate weights.
Key results
- On WebKB, GraphMETRO reaches 41.11% accuracy vs 24.61% for EERM (about 67% relative improvement).
- On Twitch, GraphMETRO reaches 53.50 ROC-AUC vs 51.34 for EERM (4.2% improvement); on SST2 it reaches 81.87% vs 81.67% for GroupDRO.
- On synthetic shifts, GraphMETRO averages 81.08% on DBLP vs 77.88% for ERM and 76.36% vs 72.29% on CiteSeer; the paper reports ~4.6% average gain over ERM.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use GraphMETRO instead of EERM | Graph neural networks · WebKB | accuracy: 24.61 ± 4.86 -> 41.11 | Helped |
| use GraphMETRO instead of EERM | Graph neural networks · Twitch | ROC-AUC: 51.34 ± 1.41 -> 53.50 | Helped |
| use GraphMETRO instead of GroupDRO | Graph neural networks · Twitter | accuracy: 56.65 ± 1.72 -> 57.24 | Helped |
| use GraphMETRO instead of GroupDRO | Graph neural networks · SST2 | accuracy: 81.67 ± 0.45 -> 81.87 | No clear effect |
| use GraphMETRO instead of ERM-Aug | GAT on synthetic node-classification datasets · DBLP average | accuracy: 78.63 -> 81.08 | Helped |
| use GraphMETRO instead of ERM-Aug | GAT on synthetic node-classification datasets · CiteSeer average | accuracy: 72.41 -> 76.36 | Helped |
| use GraphMETRO instead of ERM-Aug | GAT on synthetic graph-classification datasets · IMDB-MULTI average | accuracy: 45.58 -> 47.82 | Helped |
| use GraphMETRO instead of ERM-Aug | GAT on synthetic graph-classification datasets · REDDIT-BINARY average | accuracy: 67.56 -> 68.51 | Helped |
| add the gating-model loss L1 (ablation) | GraphMETRO · WebKB | accuracy: 23.22 -> 41.11 | Helped |
| share the GNN encoder across experts and use individual MLPs (ablation) | GraphMETRO · WebKB | accuracy: 41.11 -> 31.14 | Hurt |
| add the alignment term (λ=1) (ablation) | GraphMETRO · WebKB | accuracy: 18.79 -> 41.11 | Helped |
| increase the number of stochastic transform functions from 2 to 7 (ablation) | GraphMETRO · WebKB | accuracy: 42.4% to 31.9% | Hurt |
Surprising findings
- Removing the alignment term (lambda=0) drops WebKB accuracy from 41.11 to 18.79, showing alignment to the reference model is critical.
- Increasing the number of stochastic transform functions generally degrades performance (WebKB drops from 42.4% to 31.9%).
Limitations
- Performance depends on the predefined stochastic transform functions; unknown real-world shifts may not be captured.
- Gating model expressiveness limits performance as more transform functions are added; cost grows with number of experts.
- Theoretical guarantees rely on approximate and loosely justified arguments (e.g., Jensen-based bound).
Experimental setup
Datasets: WebKB Twitch Twitter GraphSST2 DBLP CiteSeer IMDB-MULTI REDDIT-BINARY
Benchmarks: WebKB Twitch Twitter GraphSST2 DBLP CiteSeer IMDB-MULTI REDDIT-BINARY
Metrics: accuracy ROC-AUC
Baselines: ERM DANN IRM VREx GroupDRO Deep Coral SRGNN EERM OODGAT DIR G-Mixup GSAT
Builds on: mixture-of-experts invariant risk minimization GOOD benchmark GSAT CIGA
Closest papers by meaning
- Learning Divergence Fields for Shift-Robust Graph Representations (ICML 2024 · similarity 0.85)
- Improving Out-of-Distribution Generalization in Graphs via Hierarchical Semantic Environments (CVPR 2024 · similarity 0.84)
- Virtual Nodes Can Help: Tackling Distribution Shifts in Federated Graph Learning (AAAI 2025 · similarity 0.84)
- Matcha: Mitigating Graph Structure Shifts with Test-Time Adaptation (ICLR 2025 · similarity 0.84)
- Graph Classification via Reference Distribution Learning: Theory and Practice (NeurIPS 2024 · similarity 0.84)
- Out-of-Distribution Generalization on Graphs via Progressive Inference (AAAI 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.