Research map / Graph learning / Graph self-supervised learning
Leveraging Large Language Models for Node Generation in Few-Shot Learning on Text-Attributed Graphs
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes LLM4NG, a plug-and-play paradigm that prompts ChatGPT to generate labeled paper nodes per class, links them with a trained edge predictor, and trains GCN/GAT on the augmented text-attributed graph for few-shot node classification.
Problem
Few-shot and zero-shot node classification on text-attributed graphs, where scarce labeled nodes give insufficient supervision for GNNs.
Approach
Use an LLM to generate class-conditioned sample texts from label names as new labeled nodes, embed them with a sentence encoder, connect them to the raw graph via an edge predictor trained on existing edges plus cosine-similarity filtering, and then train any GNN on the augmented graph without changing the raw data.
Key results
- On ogbn-arxiv 1-shot, LLM4NG+GCN reaches 56.83% vs 31.42% for the best baseline (GCN with MiniLM), a reported 76.55% relative improvement.
- On PubMed 0-shot, LLM4NG+GCN reaches 75.36%, above the best 5-shot baseline of 73.76%.
- With TAPE+LLM4NG combined, ogbn-arxiv 5-shot accuracy reaches 61.53%; LLM4NG uses 400 API calls versus 169,343 for TAPE.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add LLM-generated labeled nodes and graph connections | GCN · ogbn-arxiv 1-shot | accuracy: +76.55% | Helped |
| add LLM-generated labeled nodes and graph connections | LLM4NG+GCN · Cora 1-shot | accuracy: 76.48(0.90) vs 67.94(4.09) | Helped |
| add LLM-generated labeled nodes and graph connections | LLM4NG+GCN · Pubmed 1-shot | accuracy: 75.06(2.56) vs 64.88(7.84) | Helped |
| add LLM-generated labeled nodes and graph connections | LLM4NG+GCN · ogbn-arxiv 1-shot | accuracy: 56.83(1.82) vs 31.42(3.89) | Helped |
| add the edge predictor to connect generated nodes to raw graph nodes (ablation) | LLM4NG+GCN · Cora 1-shot | accuracy: 75.12(1.60) -> 76.48(0.90) | Helped |
| add the edge predictor to connect generated nodes to raw graph nodes (ablation) | LLM4NG+GAT · Cora 1-shot | accuracy: 74.38(2.03) -> 75.90(1.86) | Helped |
| add the edge predictor to connect generated nodes to raw graph nodes (ablation) | LLM4NG+GCN · Pubmed 1-shot | accuracy: 74.82(2.45) -> 75.06(2.56) | Helped |
| add the edge predictor to connect generated nodes to raw graph nodes (ablation) | LLM4NG+GAT · Pubmed 1-shot | accuracy: 74.38(2.03) -> 75.98(2.82) | Helped |
| add the edge predictor to connect generated nodes to raw graph nodes (ablation) | LLM4NG+GCN · ogbn-arxiv 1-shot | accuracy: 55.89(1.30) -> 56.83(1.82) | Helped |
| add the edge predictor to connect generated nodes to raw graph nodes (ablation) | LLM4NG+GAT · ogbn-arxiv 1-shot | accuracy: 55.18(2.28) -> 56.29(2.65) | Helped |
| add a graph-task description to the label-conditioned prompt (ablation) | LLM4NG-P1 · Cora 0-shot | accuracy: 71.0 -> 67.4 | Hurt |
| add a graph-task description to the label-conditioned prompt (ablation) | LLM4NG-P1 · Cora 1-shot | accuracy: 73.9 -> 74.8 | Helped |
Surprising findings
- Generated samples with no real-world counterpart still improve accuracy, which the authors attribute to capturing domain knowledge about labels rather than authenticity.
- Adding the edge predictor is not always beneficial: at some shot counts the w/o A variant performs comparably or better (e.g., LLM4NG+GAT w/o A on Cora 10-shot ties or matches).
Limitations
- Relies on a closed-source LLM (gpt-3.5-turbo) for generation, so reproducibility depends on API access.
- Generated samples may be pseudo-samples without real-world existence, and quality is evaluated mainly via downstream accuracy.
- Experiments are limited to three citation datasets and two GNN backbones.
Experimental setup
Datasets: Cora PubMed ogbn-arxiv
Benchmarks: Cora PubMed ogbn-arxiv
Metrics: classification accuracy
Baselines: GCN GAT DGI MVGRL GRACE Mixup DropEdge Meta-PN CGPN TAPE
Builds on: GCN GAT TAPE Meta-PN CGPN
Closest papers by meaning
- Label-free Node Classification on Graphs with Large Language Models (LLMs) (ICLR 2024 · similarity 0.91)
- LLMs as Zero-shot Graph Learners: Alignment of GNN Representations with LLM Token Embeddings (NeurIPS 2024 · similarity 0.91)
- Generalization Principles for Inference over Text-Attributed Graphs with Large Language Models (ICML 2025 · similarity 0.90)
- Dynamic Bundling with Large Language Models for Zero-Shot Inference on Text-Attributed Graphs (NeurIPS 2025 · similarity 0.90)
- TGLsta: Low-resource Textual Graph Learning with Semantic and Topological Awareness via LLMs (AAAI 2025 · similarity 0.90)
- Preference-driven Knowledge Distillation for Few-shot Node Classification (NeurIPS 2025 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.