Research map / Graph learning / Graph self-supervised learning

A Unified Graph Language Model for Multi-Domain Multi-Task Graph Alignment Instruction Tuning

NeurIPS 2026 · Graph learning › Graph self-supervised learning · approach: LLMs and graph foundation models · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes UniGraphLM, a graph language model that pretrains a multi-scale GraphSAGE encoder with a domain-aware reweighted CLIP-style contrastive loss on graph-text pairs, then aligns it to Vicuna-7B via curriculum projector tuning weighted by gradient-norm-based domain difficulty.

Problem

Aligning GNN-encoded graph representations from multiple domains and task types (node, edge, graph level) with an LLM token space to produce unified graph tokens that generalize across diverse graph data, while handling varying alignment difficulty across datasets.

Approach

Combines a label-free graph-text contrastive pretraining of a multi-scale GNN encoder, with inter-domain distance-based weights on negatives, and a curriculum that reweights domains in instruction tuning by an EMA of projector gradient norms.

Key results

• Multi-domain multi-task: UniGraphLM averages 81.95% across 8 datasets vs. 75.22% for the strongest baseline TEA-GLM (+6.73 points).
• Cross-domain (Arxiv-trained, Cora/PubMed/Wiki-CS): 55.17%, 77.31%, 63.93%, outperforming all baselines.
• Cross-task (Arxiv-trained, evaluated on FB15K237): 55.27% vs. 16.93% for GOFA; WN18RR 24.12% vs. 22.65% for TEA-GLM.

Surprising findings

• Pure LLM baselines (Vicuna-7B, LLaMA-2-7B) are far weaker than GLMs, e.g. LLaMA-2-7B averages 29.29%.

Limitations

• Focuses on conventional classification tasks (node, edge, graph) over a moderate set of datasets, without complex graph reasoning.
• Baselines like GraphGPT and LLaGA were only pretrained/evaluated on node-level datasets, and GOFA uses checkpoints with a different LLM backbone (Mistral-7B).
• Class-imbalanced molecular datasets required balanced subsampling, and repeated sampling was used for small datasets.

Experimental setup

Datasets: Cora PubMed Arxiv Wiki-CS WN18RR FB15K237 ChemHIV ChemPCBA

Benchmarks: Cora PubMed Wiki-CS Arxiv WN18RR FB15K237 ChemPCBA ChemHIV

Metrics: ACC% F1% AUC%

Baselines: Vicuna-7B LLaMA-2-7B GraphGPT LLaGA GOFA TEA-GLM

Builds on: GraphGPT LLaGA TEA-GLM CLIP GraphCLIP

Closest papers by meaning

• UniGTE: Unified Graph–Text Encoding for Zero-Shot Generalization across Graph Tasks and Domains (NeurIPS 2025 · similarity 0.91)
• Multi-View Empowered Structural Graph Wordification for Language Models (AAAI 2025 · similarity 0.91)
• Towards Graph Foundation Models: Learning Generalities Across Graphs via Task-Trees (ICML 2025 · similarity 0.90)
• Large Language Model Meets Graph Neural Network in Knowledge Distillation (AAAI 2025 · similarity 0.89)
• LLMs as Zero-shot Graph Learners: Alignment of GNN Representations with LLM Token Embeddings (NeurIPS 2024 · similarity 0.89)
• $\texttt{G1}$: Teaching LLMs to Reason on Graphs with Reinforcement Learning (NeurIPS 2025 · similarity 0.88)