Research map / Graph learning / Graph self-supervised learning
ReGCL: Rethinking Message Passing in Graph Contrastive Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Analyzes gradient conflicts between GNN message passing and InfoNCE in node-level graph contrastive learning, and proposes ReGCL with gradient-guided structure learning and gradient-weighted InfoNCE.
Problem
Node-level graph contrastive learning with GNN encoders suffers a conflict: message passing pulls neighbors together while InfoNCE pushes them apart, so some samples receive opposing gradients.
Approach
Derives gradients of InfoNCE with respect to input features (including message passing) to locate conflicting samples, then rewires the graph via a gradient-guided structure learning module (higher-order structure plus stop-gradient edge re-weighting) and down-weights likely false negatives with a gradient-weighted InfoNCE loss.
Key results
- ReGCL reaches 84.8% on Cora, 74.3% on Citeseer and 83.9% on Pubmed, outperforming the best baselines in Table 1 (e.g., CCA-SSG 84.2, 73.1, 81.6).
- The paper reports an average 5.1% gain over baselines and a 2.9% gain over GRACE, which it treats as an ablation-like version of ReGCL.
- Ablations show removing GGSL costs up to 1.1%, removing GW-NCE costs up to 1.7%, and removing higher-order structure costs about 2.0% on average.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use ReGCL instead of CCA-SSG | Node classification · Cora | accuracy: 84.2±0.4 -> 84.8±0.1 | Helped |
| Use ReGCL instead of CCA-SSG | Node classification · Citeseer | accuracy: 73.1±0.3 -> 74.3±0.3 | Helped |
| Use ReGCL instead of CCA-SSG | Node classification · Pubmed | accuracy: 81.6±0.4 -> 83.9±0.3 | Helped |
| Use ReGCL instead of CCA-SSG | Node classification · Photo | accuracy: 93.1±0.1 -> 92.6±0.3 | Hurt |
| Use ReGCL instead of CCA-SSG | Node classification · CS | accuracy: 93.3±0.2 -> 93.7±0.3 | Helped |
| Remove gradient-guided structure learning (GGSL) (ablation) | ReGCL · node classification benchmarks | accuracy | Hurt |
| Replace gradient-weighted InfoNCE with normal InfoNCE (ablation) | ReGCL · node classification benchmarks | accuracy | Hurt |
| Remove both GGSL and GW-NCE (ablation) | ReGCL · node classification benchmarks | accuracy | Hurt |
| Remove higher-order structure enhancement from GGSL (ablation) | ReGCL · node classification benchmarks | accuracy | Hurt |
| Keep original graph edges instead of re-wiring them (ablation) | ReGCL · node classification benchmarks | accuracy | Hurt |
| Increase the neighbor order beyond the first order (ablation) | ReGCL · node classification benchmarks | accuracy: 1.7% | Helped |
| Use an overly large neighbor order k (ablation) | ReGCL · node classification benchmarks | accuracy | Hurt |
Surprising findings
- Removing higher-order structure learning on Photo and CS hurts more than removing both GGSL and GW-NCE together, according to the authors.
Limitations
- Performance declines when the higher-order order k is too large, due to over-smoothing.
- Adding too many edges through the threshold delta increases conflicts and hurts performance.
- Evaluated only on five node classification datasets with a single-layer GCN analysis.
Experimental setup
Datasets: Cora Citeseer Pubmed Amazon Photo Coauthor CS
Benchmarks: Cora Citeseer Pubmed Amazon Photo Coauthor CS
Metrics: test accuracy
Baselines: GCN GAT DeepWalk GAE DGI GRACE MVGRL BGRL GCA CCA-SSG GRADE
Builds on: GRACE InfoNCE GCN
Closest papers by meaning
- Does GCL Need a Large Number of Negative Samples? Enhancing Graph Contrastive Learning with Effective and Efficient Negative Sampling (AAAI 2025 · similarity 0.93)
- A New Mechanism for Eliminating Implicit Conflict in Graph Contrastive Learning (AAAI 2024 · similarity 0.93)
- Beyond Homophily: Graph Contrastive Learning with Macro-Micro Message Passing (AAAI 2025 · similarity 0.92)
- S3GCL: Spectral, Swift, Spatial Graph Contrastive Learning (ICML 2024 · similarity 0.91)
- Exploitation of a Latent Mechanism in Graph Contrastive Learning: Representation Scattering (NeurIPS 2024 · similarity 0.91)
- Perfect Alignment May be Poisonous to Graph Contrastive Learning (ICML 2024 · similarity 0.91)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.