Research map / Graph learning / Graph self-supervised learning
Equivariance Everywhere All At Once: A Recipe for Graph Foundation Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a triple-symmetry (node, feature, label permutation) recipe for node-level graph foundation models, with a universal approximation result and TS-GNNs that wrap any MPNN aggregation, showing zero-shot transfer trained on a single graph (Cora).
Problem
Designing graph foundation models for node-level tasks that generalize zero-shot across arbitrary graphs with varying feature sets and label sets, without a shared feature vocabulary.
Approach
Identifies label permutation-equivariance and feature permutation-invariance (alongside node permutation-equivariance) as required symmetries, characterizes all linear equivariant layers under S_N x S_F x S_C, proves universality of the resulting networks, and turns any MPNN aggregation into a triple-symmetry GFM via label inpainting with least-squares feature-label mixing.
Key results
- TS-Mean achieves 65.78% average zero-shot accuracy over 28 datasets when trained only on cora, versus 65.56% for GraphAny and 65.50% for end-to-end MeanGNN.
- TS-GAT averages 65.60% zero-shot, slightly above GraphAny (65.56%) and on par with end-to-end GAT (65.55%).
- In the training-set scaling experiment, zero-shot accuracy of TS-Mean improves steadily as training graphs grow from 1 to 9, while GraphAny's performance stays unchanged.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use TS-Mean instead of MeanGNN for zero-shot transfer from Cora | MeanGNN · Average (28 graphs) | accuracy: 65.50 -> 65.78 | Helped |
| Use TS-GAT instead of GAT for zero-shot transfer from Cora | GAT · Average (28 graphs) | accuracy: 65.55 -> 65.60 | No clear effect |
| Use TS-Mean for zero-shot transfer instead of GraphAny | Zero-shot graph models · Average (28 graphs) | accuracy: 65.56 -> 65.78 | Helped |
| Increase the number of training graphs | TS-Mean · Zero-shot evaluation across 20 datasets | accuracy | Helped |
| Increase the number of training graphs | GraphAny · Zero-shot evaluation across 20 datasets | accuracy | No clear effect |
Surprising findings
- GraphAny's zero-shot performance does not improve as more training graphs are added, unlike TS-Mean.
- The authors claim a subtle flaw in the universality proof of Theorem 3 of prior DSS-style work for arbitrary feature-permutation subgroups H, restricting the guarantee to H=S_F.
Limitations
- Recipe is currently limited to node-level tasks; no edge-level or graph-level tasks.
- Node regression not evaluated due to lack of node-regression datasets.
- Zero-shot gains over GraphAny are small on average (about 0.2 points) and the scaling claim is not formally established as a scaling law.
Experimental setup
Datasets: cora roman-empire amazon-ratings minesweeper questions tolokers citeseer pubmed chameleon squirrel cornell wisconsin
Benchmarks: roman-empire amazon-ratings minesweeper questions tolokers cora citeseer pubmed chameleon squirrel cornell wisconsin
Metrics: test accuracy
Baselines: GraphAny MeanGNN GAT
Builds on: DeepSets Deep Sets for Symmetric Elements (DSS) GraphAny GraphSAGE GAT
Closest papers by meaning
- REVISITING MULTI-PERMUTATION EQUIVARIANCE THROUGH THE LENS OF IRREDUCIBLE REPRESENTATIONS (ICLR 2025 · similarity 0.89)
- Dissecting the Failure of Invariant Learning on Graphs (NeurIPS 2024 · similarity 0.88)
- Equivariant Machine Learning on Graphs with Nonlinear Spectral Filters (NeurIPS 2024 · similarity 0.88)
- Subgraph Invariant Learning Towards Large-Scale Graph Node Classification (AAAI 2025 · similarity 0.88)
- A Flexible, Equivariant Framework for Subgraph GNNs via Graph Products and Graph Coarsening (NeurIPS 2024 · similarity 0.88)
- Empowering Graph Invariance Learning with Deep Spurious Infomax (ICML 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.