Research map / Graph learning / Graph self-supervised learning
Rethinking Graph Masked Autoencoders through Alignment and Uniformity
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Builds a theoretical bridge showing GraphMAE's node-level reconstruction implicitly performs context-level graph contrastive learning, then proposes AUG-MAE with easy-to-hard adversarial masking and an explicit uniformity regularizer, improving node and graph classification.
Problem
GraphMAE works well empirically but lacks theoretical understanding, and its connection to graph contrastive learning is unknown; its alignment is limited by random masking and its uniformity is not guaranteed (partial dimensional collapse).
Approach
Shows that the SCE reconstruction loss of GraphMAE is lower-bounded by a context-level alignment loss (so it implicitly does context-level GCL), then adds an adversarial GNN-based mask generator with an easy-to-hard schedule for harder positive pairs, plus an explicit uniformity loss on representations.
Key results
- AUG-MAE reaches 84.3% on Cora, 81.4% on PubMed, and 96.1% on Reddit node classification, outperforming GraphMAE on all nine node datasets except CiteSeer where it is tied-level (73.2 vs 73.1).
- Average rank on node classification is 1.22 versus 3.00 for GraphMAE; on graph classification AUG-MAE has average rank 1.83 versus 3.00 for GraphMAE.
- Ablation shows adversarial masking, easy-to-hard training, and the uniformity regularizer each contribute, with the uniformity regularizer improving both node and graph tasks.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add AUG-MAE alignment and uniformity components | GraphMAE · Cora | accuracy: 84.0 $\pm$ 0.6 -> 84.3 $\pm$ 0.4 | Helped |
| add AUG-MAE alignment and uniformity components | GraphMAE · CiteSeer | accuracy: 73.1 $\pm$ 0.4 -> 73.2 $\pm$ 0.4 | Mixed |
| add AUG-MAE alignment and uniformity components | GraphMAE · Reddit | accuracy: 95.8 $\pm$ 0.4 -> 96.1 $\pm$ 0.1 | Helped |
| add AUG-MAE alignment and uniformity components | GraphMAE · IMDB-B | accuracy: 75.30 $\pm$ 0.59 -> 75.56 $\pm$ 0.61 | Helped |
| add AUG-MAE alignment and uniformity components | GraphMAE · MUTAG | accuracy: 88.19 $\pm$ 1.26 -> 88.28 $\pm$ 0.98 | Mixed |
| add adversarial masking to GraphMAE (ablation) | AUG-MAE · Cora | accuracy: 84.04 $\pm$ 0.58 -> 84.30 $\pm$ 0.75 | Helped |
| add uniformity regularizer to GraphMAE (ablation) | AUG-MAE · Cora | accuracy: 84.04 $\pm$ 0.58 -> 84.24 $\pm$ 0.58 | Helped |
| add adversarial masking without easy-to-hard training (ablation) | AUG-MAE · IMDB-B | accuracy: 75.30 $\pm$ 0.59 -> 74.32 $\pm$ 0.75 | Hurt |
| add easy-to-hard training alongside adversarial masking (ablation) | AUG-MAE · IMDB-B | accuracy: 74.32 $\pm$ 0.75 -> 75.20 $\pm$ 0.88 | Helped |
| add uniformity regularizer to adversarial masking (ablation) | AUG-MAE · IMDB-B | accuracy: 74.32 $\pm$ 0.75 -> 74.56 $\pm$ 0.58 | Helped |
| increase uniformity regularizer weight above 1e-1 | AUG-MAE · most datasets | accuracy | Hurt |
| vary easy-to-hard parameter eta in [0.5,1.5] | AUG-MAE · most datasets | accuracy | No clear effect |
Surprising findings
- On MUTAG and REDDIT-B, AUG-MAE beats GraphMAE but still trails some GCL methods, which the authors attribute to generative methods focusing on context-level information.
- Uniformity regularizer impairs accuracy when its weight exceeds 1e-1, showing excessive uniformity harms distinguishability.
Limitations
- On some graph classification datasets (MUTAG, REDDIT-B), AUG-MAE does not outperform strong GCL baselines.
- Theoretical analysis relies on Assumption 4.1 (a pseudo-inverse encoder with bounded reconstruction error) that is idealized.
- Hyperparameters (lambda1, lambda2, alpha0, alphaT, eta) are tuned per dataset over broad grids.
Experimental setup
Datasets: Cora CiteSeer PubMed Ogbn-arxiv PPI Reddit CoraFull Flickr WikiCS IMDB-B IMDB-M PROTEINS
Benchmarks: Cora CiteSeer PubMed Ogbn-arxiv PPI Reddit CoraFull Flickr WikiCS IMDB-B IMDB-M PROTEINS
Metrics: accuracy Micro-F1 average rank l2 distance between positive pairs
Baselines: DGI MVGRL GRACE BGRL InfoGCL CCA-SSG Graph2vec InfoGraph GraphCL JOAO GCC SeeGera
Builds on: GraphMAE alignment and uniformity Gumbel-Softmax GRACE
Closest papers by meaning
- Teacher-guided Edge Discriminator for Personalized Graph Masked Autoencoder (AAAI 2025 · similarity 0.91)
- Self-supervised Masked Graph Autoencoder via Structure-aware Curriculum (ICML 2025 · similarity 0.90)
- S3GCL: Spectral, Swift, Spatial Graph Contrastive Learning (ICML 2024 · similarity 0.90)
- Does GCL Need a Large Number of Negative Samples? Enhancing Graph Contrastive Learning with Effective and Efficient Negative Sampling (AAAI 2025 · similarity 0.89)
- Beyond Homophily: Graph Contrastive Learning with Macro-Micro Message Passing (AAAI 2025 · similarity 0.89)
- ReGCL: Rethinking Message Passing in Graph Contrastive Learning (AAAI 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.