Research map / Large language models / Retrieval and knowledge

A Systematic Exploration of Knowledge Graph Alignment with Large Language Models in Retrieval Augmented Generation

AAAI 2025 · Large language models › Retrieval and knowledge · approach: RAG robustness and evaluation · 9 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Systematically studies KG alignment with LLMs (KGA) in KG-RAG, splitting it into graph transformation and linearization phases, and runs over 3,500 experiments across 15 LLMs and three KGQA datasets.

Problem

KG+RAG methods treat the conversion of retrieved knowledge graphs into LLM input text (KG alignment, KGA) as a simple step, leaving unclear which factors matter, how LLMs understand linearized KGs, and how to exploit KGA to improve KG+RAG.

Approach

Decomposes KGA into a graph-to-graph phase (81 graph features at node, edge, and full-graph levels) and a graph-to-text linearization phase (13 formats, 13 orders, 14 templates), then uses feature analysis, mechanistic interpretability (information flow routes), and combining the best factors to improve KGQA.

Key results

• Combining the best formats, orders and templates improves KG+RAG on GraphextQA by 7.3% on average across four LLMs.
• Flat triple formats achieve the best average performance, and templates with special token separators outperform others by a 10.7-point gap between best and worst.
• Adding reverse edges or a virtual global node, which raise sub-KG centrality, improves performance on several models (e.g., Llama-3-8b from 77.40 to 88.20 with reverse edges).

Measured effects reported in the paper

Surprising findings

• Orders are model-dependent, with no single optimal order across all LLMs, contrary to prior practice of descending similarity ranking.
• Separator tokens in KG linearization are processed by distinct circuits from entities and natural words, acting as summary (BOS-like) positions.

Limitations

• Results on ChatGPT and GPT-4o use only a 10% random sample of data.
• KG-to-Text MVP model is trained on specific datasets and has low generalizability.
• Evaluation limited to KGQA datasets and Hits@1 metric.

Experimental setup

Datasets: GraphextQA ComplexWebQuestions WebQuestionsSP

Benchmarks: GraphextQA ComplexWebQuestions WebQuestionsSP

Metrics: Hits@1 Rank-biased Overlap Z-score of Hits@1

Baselines: MVP flat triple linearization Retrieve-Rewrite-Answer

Builds on: Information Flow Routes StructGPT MVP LoRA Think-on-Graph

Closest papers by meaning

• Simple is Effective: The Roles of Graphs and Large Language Models in Knowledge-Graph-Based Retrieval-Augmented Generation (ICLR 2025 · similarity 0.94)
• Can Knowledge-Graph-based Retrieval Augmented Generation Really Retrieve What You Need? (NeurIPS 2025 · similarity 0.92)
• Think-on-Graph 2.0: Deep and Faithful Large Language Model Reasoning with Knowledge-guided Retrieval Augmented Generation (ICLR 2025 · similarity 0.92)
• Graph-KV: Breaking Sequence via Injecting Structural Biases into Large Language Models (NeurIPS 2025 · similarity 0.91)
• GFM-RAG: Graph Foundation Model for Retrieval Augmented Generation (NeurIPS 2025 · similarity 0.91)
• KGGen: Extracting Knowledge Graphs from Plain Text with Language Models (NeurIPS 2025 · similarity 0.90)