Research map / Large language models / Retrieval and knowledge

StructRAG: Boosting Knowledge Intensive Reasoning of LLMs via Inference-time Hybrid Information Structurization

ICLR 2025 · Large language models › Retrieval and knowledge · approach: RAG robustness and evaluation · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes StructRAG, which routes each knowledge-intensive task to an optimal structure type (table, graph, algorithm, catalogue, chunk), reconstructs documents into that structure with an LLM, and answers via question decomposition over it, with the router trained by DPO on synthetic preference pairs.

Problem

Standard RAG struggles with knowledge-intensive reasoning where useful information is scattered across many long documents and must be integrated for global reasoning.

Approach

Route each task to a task-appropriate structured knowledge format (table, graph, algorithm, catalogue, chunk), restructure raw documents into that format at inference time, and answer by decomposing the question over the structure; the router is trained with DPO on synthetic preference data built by a synthesize-simulate-judge pipeline.

Key results

• On Loong overall, StructRAG reaches 60.38 LLM score and 0.23 EM, versus 60.11 / 0.29 for Long-context and 46.11 / 0.23 for RAG among the baselines reported in the same table.
• On the longest Loong set (200K-250K tokens), StructRAG scores 51.42 LLM score versus 29.29 for RAG, with the gap widening as document length grows.
• On Podcast Transcripts, StructRAG wins against GraphRAG with an average win rate of 53.00% and against Long-context with 95.75%.

Measured effects reported in the paper

Surprising findings

• A Qwen2-7B router trained with DPO outperforms the 72B model with few-shot prompting at selecting the structure type.
• Removing the router causes the largest performance drop (60.38 to 45.33), larger than removing the utilizer (to 55.94).

Limitations

• Structurization is probabilistic, so wording changes (e.g., number formatting) reduce exact-match rates despite high LLM scores.
• Heavy reliance on Qwen2-72B for structurization and utilization makes the pipeline costly and slower than RQ-RAG.
• Evaluation is limited to Loong and Podcast Transcripts, with LLM-based judging.

Experimental setup

Datasets: Loong Podcast Transcripts

Benchmarks: Loong Podcast Transcripts

Metrics: LLM score exact match head-to-head win rate

Baselines: Long-Context RAG RQ-RAG GraphRAG

Builds on: DPO GraphRAG RQ-RAG

Closest papers by meaning

• Retrieval is Not Enough: Enhancing RAG through Test-Time Critique and Optimization (NeurIPS 2025 · similarity 0.90)
• From Complex to Atomic: Enhancing Augmented Generation via Knowledge-Aware Dual Rewriting and Reasoning (ICML 2025 · similarity 0.89)
• Simple is Effective: The Roles of Graphs and Large Language Models in Knowledge-Graph-Based Retrieval-Augmented Generation (ICLR 2025 · similarity 0.89)
• Long-Context LLMs Meet RAG: Overcoming Challenges for Long Inputs in RAG (ICLR 2025 · similarity 0.88)
• Inference Scaling for Long-Context Retrieval Augmented Generation (ICLR 2025 · similarity 0.88)
• Cooperative Retrieval-Augmented Generation for Question Answering: Mutual Information Exchange and Ranking by Contrasting Layers (NeurIPS 2025 · similarity 0.88)