Research map / Graph learning / Graph self-supervised learning
GNN-Transformer Cooperative Architecture for Trustworthy Graph Contrastive Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes GTCA, an augmentation-free graph contrastive learning method that uses a GCN and a linear Transformer (NodeFormer) as two encoders and selects positives from the intersection of their kNN neighborhoods and topological kNN.
Problem
Graph contrastive learning relies on random augmentations that can disturb graph semantics, and GNN encoders suffer from over-smoothing and over-squashing and miss long-range dependencies, while InfoNCE uses only one positive per anchor.
Approach
Replace graph augmentation with two encoder views (GCN and NodeFormer) plus a topology view; positives for each node are the intersection of the GCN kNN, NodeFormer kNN and topological kNN sets, and a multi-positive contrastive loss pulls all of them together.
Key results
- GTCA reaches 82.5% on Cora, 69.7% on Wiki-CS, 92.5% on Coauthor-CS, 79.2% on Amazon-Computers and 90.5% on Amazon-Photo node classification accuracy, ranking first on three datasets and second on two.
- Removing the topology view (GTCA w/o T_i) drops accuracy to 79.8 on Cora and 75.7 on Amazon-Computers.
- The correct ratio of positive pairs exceeds 80% with the full intersection design, versus under 40% with a single graph encoder.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use GTCA instead of the strongest reported baseline | Node classification · Cora | accuracy (%): 82.4±1.5 -> 82.5±1.3 | Helped |
| Use GTCA instead of the strongest reported baseline | Node classification · Wiki-CS | accuracy (%): 70.3±1.7 -> 69.7±1.5 | Hurt |
| Use GTCA instead of the strongest reported baseline | Node classification · Coauthor-CS | accuracy (%): 92.0±0.5 -> 92.5±0.6 | Helped |
| Use GTCA instead of the strongest reported baseline | Node classification · Amazon-Computers | accuracy (%): 79.8±1.5 -> 79.2±1.4 | Hurt |
| Use GTCA instead of the strongest reported baseline | Node classification · Amazon-Photo | accuracy (%): 90.5±1.9 -> 90.5±1.2 | No clear effect |
| Add the topology structure view (ablation) | GTCA · Cora, Wiki-CS, Coauthor-CS, Amazon-Computers, Amazon-Photo | accuracy (%): 79.8 -> 82.5; 69.0 -> 69.7; 92.0 -> 92.5; 75.7 -> 79.2; 86.1 -> 90.5 | Helped |
| Use GCN-NodeFormer encoders instead of GCN-GCN encoders (ablation) | GTCA · Cora, Wiki-CS, Coauthor-CS, Amazon-Computers, Amazon-Photo | accuracy (%): 80.4 -> 82.5; 69.1 -> 69.7; 90.0 -> 92.5; 78.1 -> 79.2; 90.1 -> 90.5 | Helped |
| Use GCN-NodeFormer encoders instead of NodeFormer-NodeFormer encoders (ablation) | GTCA · Cora, Wiki-CS, Coauthor-CS, Amazon-Computers, Amazon-Photo | accuracy (%): 77.6 -> 82.5; 68.8 -> 69.7; 92.1 -> 92.5; 76.9 -> 79.2; 88.8 -> 90.5 | Helped |
| Increase embedding dimension E | GTCA · Cora and Amazon-Photo | accuracy (%) | Helped |
| Increase k from too-small values toward the reported optimal range | GTCA · Cora and Amazon-Photo | accuracy (%) | Helped |
| Increase k beyond the optimal range | GTCA · Cora and Amazon-Photo | accuracy (%) | Hurt |
| Increase λ up to 0.7 | GTCA · Cora and Amazon-Photo | accuracy (%) | Helped |
Surprising findings
- A NodeFormer-NodeFormer variant ranks second on Coauthor-CS, which the authors attribute to Transformers capturing long-range dependencies on the largest dataset.
- Removing the topology view yields the best Coauthor-CS score among variants (92.0), close to full GTCA (92.5).
Limitations
- Computational complexity remains a burden on large-scale graphs, as the authors acknowledge.
- The theoretical 'trustworthiness' claim is supported by a generalization bound and a proposition whose proof is informal.
- Evaluation is limited to node classification on five benchmark datasets with 20 random splits.
Experimental setup
Datasets: Cora Wiki-CS Coauthor-CS Amazon-Computers Amazon-Photo
Benchmarks: Cora Wiki-CS Coauthor-CS Amazon-Computers Amazon-Photo
Metrics: node classification accuracy
Baselines: GCN GAT CGPN CG3 DGI GMI MVGRL GRACE GCA SUGRL AFGRL NCLA
Builds on: GRACE BGRL AFGRL NodeFormer InfoNCE
Closest papers by meaning
- ReGCL: Rethinking Message Passing in Graph Contrastive Learning (AAAI 2024 · similarity 0.91)
- Does GCL Need a Large Number of Negative Samples? Enhancing Graph Contrastive Learning with Effective and Efficient Negative Sampling (AAAI 2025 · similarity 0.90)
- Leveraging Contrastive Learning for Enhanced Node Representations in Tokenized Graph Transformers (NeurIPS 2024 · similarity 0.89)
- S3GCL: Spectral, Swift, Spatial Graph Contrastive Learning (ICML 2024 · similarity 0.89)
- Unified Graph Augmentations for Generalized Contrastive Learning on Graphs (NeurIPS 2024 · similarity 0.89)
- Beyond Homophily: Graph Contrastive Learning with Macro-Micro Message Passing (AAAI 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.