Research map / Graph learning / Graph self-supervised learning
FUG: Feature-Universal Graph Contrastive Pre-training for Graphs with Diverse Node Features
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes FUG, a graph contrastive pre-training strategy with a learnable dimension encoder that maps node features of any shape into a unified shape via basis transformation, plus a global-uniformity loss that replaces negative sampling.
Problem
GNN encoders pre-trained on one graph dataset cannot directly transfer to graphs with different node feature dimensionalities; existing fixes require model rebuilding or lossy preprocessing (PCA, LLM reshaping, feature similarity).
Approach
Theoretically links PCA and InfoNCE contrastive learning under a shared framework, then instantiates a dimension encoder that produces a basis transformation matrix to losslessly unify feature shapes, combined with a GCN encoder and a neighbor-based positive loss plus a mean-to-origin global uniformity loss that reduces negative-sampling cost from O(n^2) to O(n).
Key results
- In cross-domain node classification, FUG-C pre-trained on one dataset and tested on others reaches accuracy close to rebuilt FUG; e.g., FUG-C (PubMed) gets 84.80 on PubMed vs. FUG (Rebuild) 85.47, and FUG-C (Computers) reaches 88.23 on Computers vs. 88.42 for rebuilt FUG.
- FUG (Rebuild) outperforms GRACE on all intra-domain datasets (e.g., Cora 84.45 vs 83.20; Computers 88.42 vs 85.36) and is competitive with BGRL and GBT.
- Ablation shows removing the dimension-encoder loss L_DE causes large drops, e.g., Photo falls from 93.07 to 57.37 and Computers from 88.42 to 50.55.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add LDE dimensional-encoding loss (ablation) | FUG · Photo | accuracy: 57.37 ± 3.38 -> 93.07 ± 0.82 | Helped |
| Add LDE dimensional-encoding loss (ablation) | FUG · Computers | accuracy: 50.55 ± 1.69 -> 88.42 ± 0.98 | Helped |
| Add LDE dimensional-encoding loss (ablation) | FUG · Cora | accuracy: 81.32 ± 2.56 -> 84.45 ± 2.45 | Helped |
| Add global-uniformity loss LRT-FUG− (ablation) | FUG · CiteSeer | accuracy: 68.44 ± 2.63 -> 72.43 ± 2.92 | Helped |
| Add global-uniformity loss LRT-FUG− (ablation) | FUG · Cora | accuracy: 83.31 ± 2.85 -> 84.45 ± 2.45 | Helped |
| Add global-uniformity loss LRT-FUG− (ablation) | FUG · Physics | accuracy: 95.18 ± 0.29 -> 95.45 ± 0.27 | Helped |
| Add neighbor-alignment loss LRT-FUG+ (ablation) | FUG · CiteSeer | accuracy: 70.48 ± 2.47 -> 72.43 ± 2.92 | Helped |
| Add neighbor-alignment loss LRT-FUG+ (ablation) | FUG · Cora | accuracy: 84.30 ± 2.88 -> 84.45 ± 2.45 | Helped |
| Add neighbor-alignment loss LRT-FUG+ (ablation) | FUG · Physics | accuracy: 95.29 ± 0.19 -> 95.45 ± 0.27 | Helped |
| Use FUG (Rebuild) instead of DGI | node classification · Cora | accuracy: 83.24 ± 2.12 -> 84.45 ± 2.45 | Helped |
| Use FUG (Rebuild) instead of ProGCL | node classification · CiteSeer | accuracy: 72.85 ± 2.17 -> 72.43 ± 2.92 | No clear effect |
| Use FUG (Rebuild) instead of BGRL | node classification · Physics | accuracy: 95.42 ± 0.41 -> 95.45 ± 0.27 | No clear effect |
Surprising findings
- Cross-domain FUG-C models pre-trained on other datasets sometimes outperform the same-dataset pre-trained model, e.g., FUG-C (Computers) beats FUG on Photo.
- Removing the dimension-encoder loss causes a severe collapse on Photo and Computers, showing the basis transformation must be trained explicitly.
Limitations
- Positive sampling assumes graph homophily, so it may perform poorly on heterophilic graphs.
- The instantiated encoder is a GCN, which handles only homogeneous graphs with weighted ordinary edges, not heterogeneous graphs.
- Does not constrain absolute semantic information in the loss.
Experimental setup
Datasets: Cora CiteSeer PubMed Photo Computers CS Physics
Benchmarks: Cora CiteSeer PubMed Photo Computers CS Physics
Metrics: node classification accuracy training time per epoch GPU memory (VRAM)
Baselines: PCA DeepWalk GRACE BGRL GBT DGI GCA ProGCL OFA GraphControl GraphACL ROSEN
Builds on: GRACE PCA InfoNCE BGRL GBT
Closest papers by meaning
- S3GCL: Spectral, Swift, Spatial Graph Contrastive Learning (ICML 2024 · similarity 0.88)
- ReGCL: Rethinking Message Passing in Graph Contrastive Learning (AAAI 2024 · similarity 0.88)
- Does GCL Need a Large Number of Negative Samples? Enhancing Graph Contrastive Learning with Effective and Efficient Negative Sampling (AAAI 2025 · similarity 0.87)
- Graph Structure Refinement with Energy-based Contrastive Learning (AAAI 2025 · similarity 0.87)
- Contrastive Auxiliary Learning with Structure Transformation for Heterogeneous Graphs (AAAI 2025 · similarity 0.87)
- HG-Adapter: Improving Pre-Trained Heterogeneous Graph Neural Networks with Dual Adapters (ICLR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.