Research map / Graph learning / Graph self-supervised learning
LLaGA: Large Language and Graph Assistant
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces LLaGA, which turns each node's structural neighborhood into a fixed-shape node sequence via two parameter-free templates and maps it into Vicuna's token space with a single trained MLP projector for multi-task graph learning.
Problem
Applying LLMs to graph-structured data is hard because describing graphs in natural language is verbose and loses structure, and GNNs are typically trained for a single task with task-specific heads.
Approach
Reorganize a node's sampled computational tree into a node-embedding sequence (Neighborhood Detail template) or hop-embedding sequence (Hop-Field Overview template), add Laplacian positional embeddings, and align these to the LLM token space with a lightweight MLP projector trained on node classification, link prediction and node description in a uniform QA format, with the LLM frozen.
Key results
- LLaGA-HO-7B reaches 89.85% node classification accuracy on Cora and 94.36% link prediction accuracy on Arxiv in the Task Expert and Classification Expert settings, beating the GNN baselines.
- In the General Model setting, LLaGA-HO-7B gets 89.81% link prediction on Pubmed versus 68.70% for GPT-3.5-Turbo and GraphGPT-Mix-7B's 58.86%.
- Zero-shot link prediction trained on Arxiv+Pubmed+Cora reaches 92.99% on Products, versus 56.73% for GCN.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add the Neighborhood Detail Template instead of using only the center-node embedding (ablation) | LLaGA · Arxiv node classification (Classification Expert) | accuracy: 73.92 -> 75.85 | Helped |
| add the Neighborhood Detail Template instead of using only the center-node embedding (ablation) | LLaGA · Products node classification (Classification Expert) | accuracy: 80.45 -> 83.58 | Helped |
| add the Neighborhood Detail Template instead of using only the center-node embedding (ablation) | LLaGA · Pubmed node classification (Classification Expert) | accuracy: 94.60 -> 95.06 | Helped |
| add the Neighborhood Detail Template instead of using only the center-node embedding (ablation) | LLaGA · Cora node classification (Classification Expert) | accuracy: 84.50 -> 87.64 | Helped |
| add the Hop-Field Overview Template instead of using only the center-node embedding (ablation) | LLaGA · Arxiv node classification (Classification Expert) | accuracy: 73.92 -> 75.99 | Helped |
| add the Hop-Field Overview Template instead of using only the center-node embedding (ablation) | LLaGA · Products node classification (Classification Expert) | accuracy: 80.45 -> 83.32 | Helped |
| add the Hop-Field Overview Template instead of using only the center-node embedding (ablation) | LLaGA · Pubmed node classification (Classification Expert) | accuracy: 94.60 -> 94.80 | Helped |
| add the Hop-Field Overview Template instead of using only the center-node embedding (ablation) | LLaGA · Cora node classification (Classification Expert) | accuracy: 84.50 -> 89.30 | Helped |
| add a node-level template instead of using only the center-node embedding (ablation) | LLaGA · Link prediction across Arxiv, Products, Pubmed, and Cora (Classification Expert) | accuracy: 89.98 -> 90.81; 91.73 -> 96.56; 78.19 -> 92.36; 83.97 -> 87.35 (ND); 89.98 -> 94.30; 91.73 -> 96.05; 78.19 -> 88.64; 83. | Helped |
Surprising findings
- Removing the template (center-node embedding only) drops link prediction on Pubmed from 92.36 to 78.19 for the ND template setting in the ablation.
- GraphGPT-Std-7B gets 1.40% zero-shot node classification on Products with node embeddings only, while LLaGA gets 13.89%.
Limitations
- Node classification zero-shot accuracy remains low (e.g. 34.69% on Cora with node embeddings only), showing limited transfer to new label sets.
- Evaluated only on text-attributed citation and e-commerce graphs with 7B-class LLMs; baselines run with limited hyperparameter details.
Experimental setup
Datasets: ogbn-Arxiv ogbn-Products Pubmed Cora
Benchmarks: ogbn-Arxiv ogbn-Products Pubmed Cora
Metrics: accuracy Sbert score description label accuracy
Baselines: GCN GraphSage GAT SGC SAGN NodeFormer GPT-3.5-Turbo GraphGPT
Builds on: Vicuna GraphGPT SimTeG LLaVA-style projector
Closest papers by meaning
- LoReC: Rethinking Large Language Models for Graph Data Analysis (NeurIPS 2026 · similarity 0.91)
- Attention Mechanisms Perspective: Exploring LLM Processing of Graph-Structured Data (ICML 2025 · similarity 0.90)
- Multi-View Empowered Structural Graph Wordification for Language Models (AAAI 2025 · similarity 0.90)
- Can Large Language Models Analyze Graphs like Professionals? A Benchmark, Datasets and Models (NeurIPS 2024 · similarity 0.90)
- How Do Large Language Models Understand Graph Patterns? A Benchmark for Graph Pattern Comprehension (ICLR 2025 · similarity 0.89)
- Label-free Node Classification on Graphs with Large Language Models (LLMs) (ICLR 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.