Research map / Graph learning / Graph self-supervised learning
Dissecting the Failure of Invariant Learning on Graphs
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Theoretically dissects why IRM and VREx fail for node-level graph OOD via an SCM, and proposes CIA (cross-environment intra-class alignment) and its environment-label-free variant CIA-LRA, with a PAC-Bayesian OOD error bound and GOOD benchmark gains.
Problem
Invariant learning methods such as IRM and VREx give marginal or no gains for node-level out-of-distribution generalization on graphs, and node-level settings usually lack environment labels.
Approach
Adds a class-conditional invariance constraint: align representations of same-class nodes across different environments so that the aggregation structure of the causal pattern is identified and spurious features are removed. CIA-LRA approximates this without environment labels by aligning local same-class node pairs weighted by differences in neighborhood label distributions.
Key results
- CIA-LRA improves the best baseline by 2.44% on GAT and 3.23% on GCN averaged over GOOD splits, reaching 61.94% (GAT) and 59.79% (GCN) average accuracy under covariate shift.
- CIA outperforms IRM and VREx on all splits and is the best among non-graph-specific methods (average 58.38% GCN covariate vs 57.12% IRM and 57.62% VREx).
- On the toy dataset with lambda=0.5, CIA's accuracy collapses to about random guessing (25%) after roughly 50 epochs, while CIA-LRA maintains high accuracy.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add class-conditional cross-environment representation alignment (CIA) | IRM/VREx-style node OOD learning · GOOD node-level OOD, all reported splits | accuracy | Helped |
| Use CIA-LRA instead of CIA, with localized, reweighted alignment and no environment labels | CIA on GOOD graph OOD benchmarks · GOOD average accuracy, GAT and GCN | accuracy: CIA-LRA improves the best baseline methods by 2.44% and 3.23% on GAT and GCN, respectively | Helped |
| Replace VREx with CIA in the EERM loss | EERM · GOOD node-level OOD | accuracy | Helped |
| Align only local same-class pairs within 1 hop rather than all pairs (ablation) | CIA · Toy concept shift, epoch 200 | accuracy: 0.253 -> 0.354; 0.250 -> 0.312 | Helped |
| Align only local same-class pairs within 1 hop rather than all pairs (ablation) | CIA · Toy concept shift, epoch 200 | variance of invariant representation: 0.0003 -> 0.2327; 0.0002 -> 0.1699 | Helped |
| Increase the alignment weight λ from 0.05 to 0.1 and 0.5 (ablation) | CIA · Toy dataset, epoch 50 | variance of invariant representation: 0.061 -> 0.039 -> 0.011 | Hurt |
| Use a large alignment weight λ=0.5 (ablation) | CIA · Toy concept shift | OOD accuracy: 25% | Hurt |
| Use localized and reweighted alignment to avoid excessive alignment | CIA-LRA compared with CIA at λ=0.5 · Toy concept shift | OOD accuracy | Helped |
| Include the HeteNLD discrepancy term r^diff in alignment weights (ablation) | CIA-LRA · Synthetic concept shift | spurious representation norm | Helped |
| Include the HeteNLD discrepancy term r^diff in alignment weights (ablation) | CIA-LRA · Synthetic covariate shift | OOD accuracy | Helped |
| Account for homophilic neighbor-ratio discrepancy r^same when weighting alignment (ablation) | CIA-LRA · Synthetic graph OOD | variance of invariant representation | Helped |
| Use too small a local alignment range (t=2) (ablation) | CIA-LRA · GOOD graph OOD benchmarks | accuracy | Hurt |
Surprising findings
- IRM and VREx offer marginal or no improvement over ERM on node-level graph OOD benchmarks, even though they provably learn invariant features in the non-graph version of the SCM.
- Excessive alignment with a large lambda causes CIA to collapse invariant representations, so aligning all same-class cross-environment pairs can hurt performance.
Limitations
- Theoretical analysis relies on a simplified linear/SGC-like GNN and a Gaussian CSBM-OOD model.
- Environment labels are needed for CIA, and CIA-LRA's gains depend on hyperparameters such as lambda and hop range t, requiring grid search.
- Experiments are limited to 3-layer GAT and GCN on GOOD datasets; EERM results on some Cora GCN splits were taken from prior work due to OOM.
Experimental setup
Datasets: Arxiv Cora CBAS WebKB GOOD benchmark synthetic 4-class toy graph
Benchmarks: GOOD Arxiv GOOD Cora GOOD CBAS GOOD WebKB
Metrics: OOD test accuracy
Baselines: ERM IRM VREx GroupDRO Deep Coral IGA MatchDG EERM SRGNN CIT CaNet Mixup
Builds on: IRM VREx MatchDG CSBM GOOD benchmark
Closest papers by meaning
- Graph Contrastive Invariant Learning from the Causal Perspective (AAAI 2024 · similarity 0.90)
- Graph Invariant Learning with Subgraph Co-mixup for Out-of-Distribution Generalization (AAAI 2024 · similarity 0.90)
- Quantifying Distributional Invariance in Causal Subgraph for IRM-Free Graph Generalization (NeurIPS 2025 · similarity 0.90)
- Out-of-Distribution Generalization on Graphs via Progressive Inference (AAAI 2025 · similarity 0.89)
- Diagnosing and Rectifying Fake OOD Invariance: A Restructured Causal Approach (AAAI 2024 · similarity 0.89)
- Enhancing Graph Invariant Learning from a Negative Inference Perspective (ICML 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.