Research map / Graph learning / Graph self-supervised learning
Harnessing Language Model for Cross-Heterogeneity Graph Knowledge Transfer
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes LMCH, which serializes heterogeneous graphs into metapath-based text corpora, fine-tunes DistilBERT on source HGs, then iteratively co-trains it with an RGCN predictor plus LM-GNN contrastive alignment for cross-heterogeneity few-shot node classification.
Problem
Few-shot node classification in heterogeneous graphs where source and target graphs have different node and edge types (cross-heterogeneity transfer), with label sparsity in the target graph.
Approach
Unify heterogeneous graphs of different schemas as metapath-based natural-language corpora so a pretrained language model can automatically learn transferable general knowledge from source HGs, instead of relying on hand-defined relation patterns; then iteratively exchange soft labels between the LM and a GNN on the target HG and align them with contrastive loss.
Key results
- LMCH outperforms the best baseline by an average of 5.16% accuracy and 6.22% Macro-F1 across 32 settings on four datasets.
- In a 3-way 3-shot DBLP setting, LMCH reaches 92.31% accuracy, above CGFL's 90.26% (2-way 3-shot) and 57.45% (3-way 3-shot on YELP-like column; table layout ambiguous).
- Ablation: removing metapath corpora (LMCH-V1) drops DBLP 3-way 3-shot accuracy to 84.88% from 92.31%.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use LMCH instead of CGFL | Cross-heterogeneity few-shot node classification · DBLP 3-way 3-shot | accuracy: 85.72 -> 92.31 | Helped |
| Use LMCH instead of CGFL | Cross-heterogeneity few-shot node classification · IMDB 3-way 3-shot | accuracy: 41.81 -> 44.92 | Helped |
| Use LMCH instead of CGFL | Cross-heterogeneity few-shot node classification · YELP 3-way 3-shot | accuracy: 49.04 -> 49.72 | Helped |
| Use LMCH instead of CGFL | Cross-heterogeneity few-shot node classification · PubMed 3-way 3-shot | accuracy: 42.04 -> 44.62 | Helped |
| Add metapath-based corpus construction rather than using node attributes only (ablation) | LMCH · DBLP 3-way 3-shot | accuracy: 83.31 -> 92.31 | Helped |
| Add metapath-based corpus construction rather than using node attributes only (ablation) | LMCH · IMDB 3-way 3-shot | accuracy: 42.86 -> 44.92 | Helped |
| Add advance LM fine-tuning before GNN-supervised LM fine-tuning (ablation) | LMCH · DBLP 3-way 3-shot | accuracy: 90.93 -> 92.31 | Helped |
| Add advance LM fine-tuning before GNN-supervised LM fine-tuning (ablation) | LMCH · IMDB 3-way 3-shot | accuracy: 44.64 -> 44.92 | Helped |
| Add GNN supervision to target LM fine-tuning (ablation) | LMCH · DBLP 3-way 3-shot | accuracy: 88.43 -> 92.31 | Helped |
| Add GNN supervision to target LM fine-tuning (ablation) | LMCH · IMDB 3-way 3-shot | accuracy: 34.94 -> 44.92 | Helped |
| Add LM-GNN contrastive alignment (ablation) | LMCH · DBLP 3-way 3-shot | accuracy: 91.45 -> 92.31 | Helped |
| Add LM-GNN contrastive alignment (ablation) | LMCH · IMDB 3-way 3-shot | accuracy: 40.09 -> 44.92 | Helped |
Surprising findings
- Node-attribute-only corpora perform much worse on IMDB (36.11% accuracy) than metapath corpora (44.92%), while the best-second method differs by dataset (random-walk on DBLP, first-order neighbors on IMDB).
Limitations
- Relies on a relatively small LM (DistilBERT); larger LLMs left for future work
- Gains depend on dataset: performance varies with corpus construction and whether datasets depend on local or long-range structure
- Experiments limited to four benchmark datasets and N-way K-shot node classification
Experimental setup
Datasets: IMDB DBLP YELP PubMed
Benchmarks: IMDB DBLP YELP PubMed
Metrics: accuracy Macro-F1
Baselines: GCN GAT RGCN HAN MAML ProtoNet GPN G-Meta GLEM LMBot CGFL
Builds on: CGFL GLEM LMBot RGCN DistilBERT
Closest papers by meaning
- HeterSEED: Semantics–Structure Decoupling for Heterogeneous Graph Learning under Heterophily (NeurIPS 2026 · similarity 0.89)
- Bootstrapping Heterogeneous Graph Representation Learning via Large Language Models: A Generalized Approach (AAAI 2025 · similarity 0.89)
- Heterogeneous Graph Neural Network on Semantic Tree (AAAI 2025 · similarity 0.88)
- Contrastive Auxiliary Learning with Structure Transformation for Heterogeneous Graphs (AAAI 2025 · similarity 0.87)
- Multi-Domain Graph Foundation Models: Robust Knowledge Transfer via Topology Alignment (ICML 2025 · similarity 0.87)
- Towards Graph Foundation Models: Learning Generalities Across Graphs via Task-Trees (ICML 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.