Research map / Large language models / Retrieval and knowledge

Graph Neural Prompting with Large Language Models

AAAI 2024 · Large language models › Retrieval and knowledge · approach: Knowledge-augmented question answering · 58 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes Graph Neural Prompting (GNP), a plug-and-play module that encodes retrieved ConceptNet/UMLS subgraphs into a soft graph prompt for frozen or LoRA-tuned FLAN-T5 models, evaluated on six commonsense and biomedical QA datasets.

Problem

Pre-trained LLMs struggle to capture and return grounded knowledge, and joint training with KGs is costly for large models; the paper asks how to inject beneficial KG knowledge into frozen or lightly tuned LLMs without noisy direct KG flattening.

Approach

Retrieve subgraphs for each question/option, encode them with a GAT, pool nodes with cross-modality attention against the text token embeddings, project to the LLM embedding space as a soft prompt, and add a DistMult-based self-supervised link prediction loss.

Key results

• With frozen FLAN-T5 xlarge (3B), GNP reaches 74.36 total accuracy vs 65.95 for Prompt Tuning, a +12.76% relative gain.
• With frozen FLAN-T5 xxlarge (11B), GNP reaches 77.84 total vs 68.55 for Prompt Tuning (+13.54%).
• LoRA + GNP improves over LoRA by +1.8% on average and matches or surpasses full fine-tuning in 10 of 12 evaluations.

Measured effects reported in the paper

Surprising findings

• Direct KG injection methods (KG Flattening, KAPING) substantially hurt frozen-LLM performance relative to LLM-only.
• Explicitly modeling relations with a relational GNN decreased performance compared with a standard GNN.

Limitations

• Relies on entity linking and subgraph retrieval quality; noisy retrieved subgraphs can introduce irrelevant entities.
• Evaluated only on multiple-choice QA with FLAN-T5 models up to 11B parameters.

Experimental setup

Datasets: OpenBookQA ARC PIQA RiddleSense PubMedQA BioASQ ConceptNet UMLS

Benchmarks: OpenBookQA ARC PIQA RiddleSense PubMedQA BioASQ

Metrics: accuracy

Baselines: LLM-only Prompt Designs KG Flattening REL KG Flattening BFS KAPING Prompt Tuning LoRA Full Fine-tuning

Builds on: Prompt Tuning LoRA KAPING graph attention network DistMult

Closest papers by meaning

• KnowGPT: Knowledge Graph based Prompting for Large Language Models (NeurIPS 2024 · similarity 0.90)
• Large Language Model Meets Graph Neural Network in Knowledge Distillation (AAAI 2025 · similarity 0.88)
• LLM Enhancers for GNNs: An Analysis from the Perspective of Causal Mechanism Identification (ICML 2025 · similarity 0.88)
• GOFA: A Generative One-For-All Model for Joint Graph Language Modeling (ICLR 2025 · similarity 0.87)
• Can Graph Learning Improve Planning in LLM-based Agents? (NeurIPS 2024 · similarity 0.87)
• Simple is Effective: The Roles of Graphs and Large Language Models in Knowledge-Graph-Based Retrieval-Augmented Generation (ICLR 2025 · similarity 0.86)