Research map / Large language models / Retrieval and knowledge
Improving Retrieval Augmented Language Model with Self-Reasoning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a self-reasoning RAG framework that trains LLaMA2 end-to-end on 2,000 GPT-4-generated self-reasoning trajectories (relevance judgment, evidence selection with citations, and analysis) to improve reliability and traceability of retrieval-augmented LMs.
Problem
Retrieval-augmented language models are hurt by irrelevant retrieved documents and often fail to explicitly cite sources, limiting reliability and traceability.
Approach
Train an LLM to generate its own reasoning trajectories in a single pass: a relevance-aware process, an evidence-aware selective process with citations and key snippets, and a trajectory analysis process before the final answer, using stage-wise masked training and quality-filtered GPT-4 data, with no external NLI/critic modules at inference.
Key results
- Self-Reasoning 13B reaches 41.4 on NQ, 57.3 on PopQA and 83.9 on FEVER, versus 35.4/56.1/60.6 for Vicuna-13B and 38.8/55.8/72.1 for Self-RAG-13B.
- On ASQA, Self-Reasoning 13B gets 72.3 citation recall, above GPT-4's 68.5, with 71.2 citation precision versus GPT-4's 75.6.
- Self-Reasoning uses 2K training samples, while Self-RAG requires 145K generator and 46K critic instances.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add RAP (ablation) | Self-Reasoning 13B · NaturalQuestion, PopQA, and FEVER | accuracy: 39.9 -> 41.4; 54.3 -> 57.3; 72.2 -> 83.9 | Helped |
| add EAP (ablation) | Self-Reasoning 13B · NaturalQuestion, PopQA, and FEVER | accuracy: 37.2 -> 41.4; 53.2 -> 57.3; 78.4 -> 83.9 | Helped |
| add TAP (ablation) | Self-Reasoning 13B · NaturalQuestion, PopQA, and FEVER | accuracy: 38.2 -> 41.4; 53.4 -> 57.3; 81.2 -> 83.9 | Helped |
| use gradual learning instead of end-to-end training without stages (ablation) | Self-Reasoning 13B · NaturalQuestion, PopQA, and FEVER | accuracy: 39.5 -> 41.4; 55.3 -> 57.3; 81.2 -> 83.9 | Helped |
| use quality-controlled training samples instead of unfiltered samples (ablation) | Self-Reasoning 13B · NaturalQuestion, PopQA, and FEVER | accuracy: 37.7 -> 41.4; 54.2 -> 57.3; 80.8 -> 83.9 | Helped |
| add self-generated trajectories | LLaMA2-13B · NaturalQuestion, PopQA, and FEVER | accuracy: 32.7 -> 38.3; 53.5 -> 54.2; 53.4 -> 79.2 | Helped |
| add self-generated trajectories | Vicuna 13B · NaturalQuestion, PopQA, and FEVER | accuracy: 35.4 -> 38.5; 56.1 -> 56.4; 60.6 -> 79.6 | Helped |
Surprising findings
- Adding the self-generated trajectories to unmodified LLaMA2-13B at inference raised FEVER accuracy from 53.4 to 79.2, without any training.
Limitations
- Training data depends on GPT-4-generated trajectories and citation tools, so label quality relies on GPT-4 and automatic citation metrics.
- Automatic NLI-based citation evaluation cannot detect partially supported citations, according to the authors.
Experimental setup
Datasets: NaturalQuestions PopQA ASQA FEVER
Benchmarks: NaturalQuestions PopQA ASQA FEVER
Metrics: accuracy EM recall citation precision citation recall
Baselines: LLaMA2 LLaMA2-Chat Vicuna LLaMA2-FT ReAct RECOMP Self-RAG GPT-4
Builds on: Self-RAG Chain-of-Thought ReAct ALCE
Closest papers by meaning
- Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection (ICLR 2024 · similarity 0.90)
- Making Retrieval-Augmented Language Models Robust to Irrelevant Context (ICLR 2024 · similarity 0.89)
- RA-DIT: Retrieval-Augmented Dual Instruction Tuning (ICLR 2024 · similarity 0.88)
- Accelerating Iterative Retrieval-augmented Language Model Serving with Speculation (ICML 2024 · similarity 0.88)
- Retrieval is Not Enough: Enhancing RAG through Test-Time Critique and Optimization (NeurIPS 2025 · similarity 0.88)
- Self-Retrieval: End-to-End Information Retrieval with One Large Language Model (NeurIPS 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.