Research map / Large language models / Retrieval and knowledge
Boosting the Potential of Large Language Models with an Intelligent Information Assistant
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes AssistRAG, a trainable information-assistant LLM plugin (ChatGLM3-6B) that manages memory and knowledge for a frozen main LLM, trained with curriculum assistant learning and DPO-based reinforced preference optimization.
Problem
Standard RAG pipelines struggle with complex multi-hop reasoning; prompt-based RAG is costly and depends on base LLM capability, while SFT-based RAG requires retraining for each new LLM and may degrade its general abilities.
Approach
A trainable assistant LLM, separate from a frozen main LLM, performs question decomposition, knowledge extraction, note-taking, memory retrieval and plan-based relevance filtering. It is trained with curriculum SFT on GPT-4-annotated data and then aligned to the main LLM's preferences via DPO using F1-based preference pairs.
Key results
- AssistRAG with ChatGPT-3.5 reaches F1 44.8 on HotpotQA, 45.6 on 2Wiki and 41.4 on Bamboogle, outperforming the best listed baselines (e.g., IRCoT 42.6 F1 on 2Wiki, Self-RAG 35.5 F1 on Bamboogle).
- Relative to Naive RAG, AssistRAG improves performance by 78%, 51% and 40% for LLaMA2-7B, ChatGLM-6B and ChatGPT respectively, indicating larger gains for weaker main LLMs.
- AssistRAG uses about 90 API tokens per question versus 1890 for IRCoT, at about 0.009 cents per question and 5.73 s inference time.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use AssistRAG instead of the strongest listed baseline | ChatGPT 3.5 · HotpotQA | F1: 43.1 -> 44.8 | Helped |
| Use AssistRAG instead of the strongest listed baseline | ChatGPT 3.5 · 2Wiki | F1: 42.6 -> 45.6 | Helped |
| Use AssistRAG instead of the strongest listed baseline | ChatGPT 3.5 · Bamboogle | F1: 38.8 -> 41.4 | Helped |
| Add note-taking rather than remove it (ablation) | AssistRAG · HotpotQA, 2Wiki, Bamboogle | F1: 40.2 -> 44.8; 42.0 -> 45.6; 39.0 -> 41.4 | Helped |
| Keep note-taking trainable rather than freeze it (ablation) | AssistRAG · HotpotQA, 2Wiki, Bamboogle | F1: 41.3 -> 44.8; 43.1 -> 45.6; 39.9 -> 41.4 | Helped |
| Add question decomposition rather than remove it (ablation) | AssistRAG · HotpotQA, 2Wiki, Bamboogle | F1: 39.5 -> 44.8; 37.8 -> 45.6; 37.0 -> 41.4 | Helped |
| Keep question decomposition trainable rather than freeze it (ablation) | AssistRAG · HotpotQA, 2Wiki, Bamboogle | F1: 41.3 -> 44.8; 40.3 -> 45.6; 37.8 -> 41.4 | Helped |
| Add knowledge extraction rather than remove it (ablation) | AssistRAG · HotpotQA, 2Wiki, Bamboogle | F1: 39.2 -> 44.8; 38.5 -> 45.6; 38.7 -> 41.4 | Helped |
| Keep knowledge extraction trainable rather than freeze it (ablation) | AssistRAG · HotpotQA, 2Wiki, Bamboogle | F1: 40.9 -> 44.8; 39.7 -> 45.6; 39.4 -> 41.4 | Helped |
| Add planning rather than omit it (ablation) | AssistRAG · HotpotQA, 2Wiki, Bamboogle | F1: 43.0 -> 44.8; 44.5 -> 45.6; 40.7 -> 41.4 | Helped |
| Use curriculum learning rather than omit it (ablation) | AssistRAG · HotpotQA, 2Wiki, Bamboogle | F1: 43.2 -> 44.8; 44.3 -> 45.6; 40.0 -> 41.4 | Helped |
| Use DPO rather than omit it (ablation) | AssistRAG · HotpotQA, 2Wiki, Bamboogle | F1: 42.5 -> 44.8; 43.2 -> 45.6; 40.5 -> 41.4 | Helped |
Surprising findings
- Gains are largest for the weakest main LLM (LLaMA2-chat 7B), suggesting the assistant mainly helps by filtering noise for less robust models.
Limitations
- Adds computational complexity and latency from running a separate assistant during inference.
- Performance depends on quality and coverage of the external knowledge base and internal memory.
- Most errors (58% on 50 HotpotQA errors) stem from insufficient knowledge retrieval.
Experimental setup
Datasets: HotpotQA 2WikiMultiHopQA Bamboogle Wikipedia dump
Benchmarks: HotpotQA 2WikiMultiHopQA Bamboogle
Metrics: Exact Match F1 Precision token usage inference time cost
Baselines: CloseBook Naive RAG ReAct IRCoT Self-Ask Self-RAG LLMLingua
Builds on: Self-RAG ReAct IRCoT DPO LLM Embedder
Closest papers by meaning
- Long-Context LLMs Meet RAG: Overcoming Challenges for Long Inputs in RAG (ICLR 2025 · similarity 0.89)
- DynamicRAG: Leveraging Outputs of Large Language Model as Feedback for Dynamic Reranking in Retrieval-Augmented Generation (NeurIPS 2025 · similarity 0.89)
- Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection (ICLR 2024 · similarity 0.88)
- RuAG: Learned-rule-augmented Generation for Large Language Models (ICLR 2025 · similarity 0.88)
- Simple is Effective: The Roles of Graphs and Large Language Models in Knowledge-Graph-Based Retrieval-Augmented Generation (ICLR 2025 · similarity 0.88)
- Retrieval is Not Enough: Enhancing RAG through Test-Time Critique and Optimization (NeurIPS 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.