Research map / Graph learning / Graph self-supervised learning
WalkLM: A Uniform Language Model Fine-tuning Framework for Attributed Graph Embedding
Yanchao Tan, Zihao Zhou, Hang Lv, Weiming Liu, Carl Yang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add graph-aware fine-tuning on random-walk text to the plain LM (DRoBERTa) (ablation) | LM (DRoBERTa) · PubMed link prediction | AUC: 60.97 -> 85.65 | Improved |
| add graph-aware fine-tuning to the plain LM (ablation) | LM (DRoBERTa) · MIMIC-III link prediction | AUC: 51.44 -> 82.15 | Improved |
| add graph-aware fine-tuning to the plain LM (ablation) | LM (DRoBERTa) · PubMed node classification | Macro-F1: 58.29 -> 60.42 | Improved |
| add graph-aware fine-tuning to the plain LM (ablation) | LM (DRoBERTa) · MIMIC-III node classification | Macro-F1: 66.25 -> 75.16 | Improved |
| combine LM with RGCN instead of RGCN alone (ablation) | RGCN · PubMed node classification | Macro-F1: 12.50 -> 13.83 | Mixed |
| combine LM with HGT instead of HGT alone (ablation) | HGT · MIMIC-III link prediction | AUC: 64.01 -> 65.01 | Improved |
| choose GPT-2 instead of DRoBERTa as the LM (ablation) | LM · PubMed node classification | Macro-F1: 58.29 -> 59.43 | Mixed |
| choose GPT-2 instead of DRoBERTa as the LM (ablation) | LM · PubMed link prediction | AUC: 60.97 -> 51.71 | Hurt |
| choose XRoBERTa instead of DRoBERTa as the LM (ablation) | LM · PubMed node classification | Macro-F1: 58.29 -> 40.10 | Hurt |
| remove the graph ID virtual node (ablation) | WalkLM · MUTAG | accuracy: 81.39 -> 79.77 | Hurt |
| masked ratio m from 0.15 to 0.05 (ablation) | WalkLM · PubMed node classification | Macro-F1: 60.42 -> 52.97 | Hurt |
| masked ratio m from 0.15 to 0.35 (ablation) | WalkLM · PubMed node classification | Macro-F1: 60.42 -> 52.22 | Hurt |
Closest papers by meaning
- Generalization Principles for Inference over Text-Attributed Graphs with Large Language Models (ICML 2025 · similarity 0.91)
- GLBench: A Comprehensive Benchmark for Graph with Large Language Models (NeurIPS 2024 · similarity 0.88)
- Harnessing Explanations: LLM-to-LM Interpreter for Enhanced Text-Attributed Graph Representation Learning (ICLR 2024 · similarity 0.88)
- Label-free Node Classification on Graphs with Large Language Models (LLMs) (ICLR 2024 · similarity 0.88)
- DTGB: A Comprehensive Benchmark for Dynamic Text-Attributed Graphs (NeurIPS 2024 · similarity 0.87)
- SSTAG: Structure-Aware Self-Supervised Learning Method for Text-Attributed Graphs (NeurIPS 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
