Research map / Graph learning / GNN architectures
Efficient Contrastive Learning for Fast and Accurate Inference on Graphs
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes GraphECL, a graph contrastive learning method that trains an MLP encoder to mimic a GNN's neighborhood structure via cross-model contrastive loss, enabling fast message-passing-free inference.
Problem
Existing graph contrastive learning methods rely on GNN encoders whose message passing makes inference latency high, while MLP-based alternatives lose substantial accuracy, and distillation methods need labels.
Approach
Couple a GNN and an MLP in a cross-model InfoNCE-style loss where the MLP representation of a node is pulled toward GNN representations of its neighbors (neighborhood positives), with intra- and inter-model negatives; at inference only the MLP is used.
Key results
- GraphECL reaches 84.25% on Cora and 94.22% on Photo transductive node classification, outperforming all listed GCL baselines.
- On Flickr GraphECL attains 48.49% vs. 46.95% for the best baseline (AF-GCL/SUGRL-range), and 69.19% on Cornell vs. 59.33% for GraphACL.
- On large-scale graphs, the MLP learned by GraphECL is about 200x faster at inference than CCA-SSG with the same number of layers, while achieving 61.45% on ogbn-papers100M.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| remove inter-model negative pairs (ablation) | GraphECL · Flickr | accuracy: 48.49±0.15 -> 42.34±0.03 | Hurt |
| remove intra-model negative pairs (ablation) | GraphECL · Flickr | accuracy: 48.49±0.15 -> 42.34±0.01 | Hurt |
| remove both intra-model and inter-model negative pairs (ablation) | GraphECL · Flickr | accuracy: 48.49±0.15 -> 40.25±0.05 | Hurt |
| use only the MLP encoder (ablation) | GraphECL · Flickr | accuracy: 48.49±0.15 -> 44.83±0.06 | Hurt |
| remove inter-model negative pairs and use only MLP encoder (ablation) | GraphECL · Flickr | accuracy: 48.49±0.15 -> 42.34±0.04 | Hurt |
| remove intra-model negative pairs and use only MLP encoder (ablation) | GraphECL · Flickr | accuracy: 48.49±0.15 -> 42.32±0.10 | Hurt |
| remove both negative-pair types and use only MLP encoder (ablation) | GraphECL · Flickr | accuracy: 48.49±0.15 -> 41.28±0.02 | Hurt |
| replace generalized contrastive loss with InfoNCE (ablation) | GraphECL · Cora | accuracy: 74.55±0.45 -> 84.25±0.05 | Hurt |
| replace generalized contrastive loss with InfoNCE (ablation) | GraphECL · Citeseer | accuracy: 67.15±0.25 -> 73.15±0.41 | Hurt |
| replace generalized contrastive loss with InfoNCE (ablation) | GraphECL · Pubmed | accuracy: 76.50±1.20 -> 82.21±0.05 | Hurt |
| replace generalized contrastive loss with InfoNCE (ablation) | GraphECL · Photo | accuracy: 91.44±0.61 -> 94.22±0.11 | Hurt |
| replace generalized contrastive loss with InfoNCE (ablation) | GraphECL · Actor | accuracy: 33.39±0.37 -> 35.80±0.89 | Hurt |
Surprising findings
- Simply replacing GNN encoders with MLP in existing GCL methods causes a large accuracy drop, and even the cross-model MLP-augmented variant loses performance versus GNN-GNN.
- An MLP-only encoder trained without the cross-model loss cannot match SOTA, so the GNN-MLP coupling is essential.
Limitations
- Relies on node attributes; graphs without informative features are not addressed.
- Theoretical guarantees are for linear probing and an idealized global minimum.
Experimental setup
Datasets: Cora Citeseer Pubmed Photo WikiCS Flickr Cornell Wisconsin Texas Actor snap-patents ogbn-arxiv
Benchmarks: Cora Citeseer Pubmed Photo WikiCS Flickr Cornell Wisconsin Texas Actor snap-patents ogbn-arxiv
Metrics: accuracy inference time
Baselines: Graph-MLP VGAE DGI GCA SUGRL BGRL CCA-SSG AF-GCL AFGRL GGD GraphACL SGCL
Builds on: InfoNCE Graph-MLP GCN GraphSAGE BGRL
Closest papers by meaning
- S3GCL: Spectral, Swift, Spatial Graph Contrastive Learning (ICML 2024 · similarity 0.91)
- ReGCL: Rethinking Message Passing in Graph Contrastive Learning (AAAI 2024 · similarity 0.89)
- Beyond Homophily: Graph Contrastive Learning with Macro-Micro Message Passing (AAAI 2025 · similarity 0.89)
- A New Mechanism for Eliminating Implicit Conflict in Graph Contrastive Learning (AAAI 2024 · similarity 0.89)
- Does GCL Need a Large Number of Negative Samples? Enhancing Graph Contrastive Learning with Effective and Efficient Negative Sampling (AAAI 2025 · similarity 0.88)
- Graph Structure Refinement with Energy-based Contrastive Learning (AAAI 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.