Research map / Large language models / Retrieval and knowledge
Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Self-RAG trains a single LM to adaptively retrieve passages on demand and critique its own generations using reflection tokens, with critic labels distilled from GPT-4 and controllable segment-level beam search at inference.
Problem
Standard retrieval-augmented generation retrieves a fixed number of passages indiscriminately, regardless of whether retrieval is needed or passages are relevant, which hurts versatility and factuality, and outputs are not guaranteed to be supported by retrieved evidence.
Approach
Train an arbitrary LM to emit special reflection tokens (retrieve, relevance, support, utility) as next-token predictions, with labels produced offline by a critic model distilled from GPT-4, and use these tokens at inference for adaptive retrieval and weighted segment-level beam search without an external critic or RLHF.
Key results
- Self-RAG 13B reaches 55.8 on PopQA and 74.5 on PubHealth, outperforming retrieval-augmented Llama2-chat-13B and Alpaca-13B on these tasks.
- On ASQA, Self-RAG 13B achieves 70.3 citation precision, exceeding retrieval-augmented ChatGPT in citation precision.
- Self-RAG 7B reaches 81.2 FActScore on biographies, above ChatGPT's 71.8 and CoVE-65B's 71.2.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Train Llama2 with Self-RAG retrieval and reflection | Llama2-7B · PopQA | accuracy: 48.7 -> 54.9 | Helped |
| Train Llama2 with Self-RAG retrieval and reflection | Llama2-7B · PubHealth | accuracy: 64.3 -> 72.4 | Helped |
| Train Llama2 with Self-RAG retrieval and reflection | Llama2-7B · ALCE-ASQA | citation precision: 5.0 -> 66.9 | Helped |
| Use Self-RAG 7B instead of ChatGPT | ChatGPT · PubHealth | accuracy: 70.1 -> 72.4 | Helped |
| Use Self-RAG 7B instead of ChatGPT | ChatGPT · PopQA | accuracy: 29.3 -> 54.9 | Helped |
| Use Self-RAG 7B instead of ChatGPT | ChatGPT · Biography generation | FactScore: 71.8 -> 81.2 | Helped |
| Use Self-RAG 7B instead of ChatGPT | ChatGPT · ALCE-ASQA | MAUVE: 68.8 -> 74.3 | Helped |
| Use Self-RAG 13B instead of ChatGPT | ChatGPT · TriviaQA-unfiltered | accuracy: 74.3 -> 69.3 | Hurt |
| Use Self-RAG 13B instead of ChatGPT | ChatGPT · ARC-Challenge | accuracy: 75.3 -> 73.1 | Hurt |
| Use Self-RAG 13B instead of retrieval-augmented ChatGPT | Ret-ChatGPT · ALCE-ASQA | citation precision: 65.1 -> 70.3 | Helped |
| Always retrieve the top one document instead of adaptive retrieval and passage selection (ablation) | Self-RAG · PopQA and ALCE-ASQA | performance | Hurt |
| Remove the support score from critique-guided beam search (ablation) | Self-RAG · ALCE-ASQA | performance | Hurt |
Surprising findings
- Self-RAG 7B sometimes outperforms its 13B variant on factual precision metrics, attributed to shorter, precisely grounded outputs.
- Standard retrieval with off-the-shelf retrievers gives little or no gain on PubHealth and ARC-Challenge compared with no retrieval.
Limitations
- Citation outputs can still be unsupported by the cited passages despite self-critique.
- Relies on GPT-4-generated labels for the critic, with lower agreement on usefulness (80%) scores.
- Retriever (Contriever) was trained mainly on knowledge-intensive tasks and may be ill-suited for open-ended instruction following.
Experimental setup
Datasets: Open-Instruct (ShareGPT, GPT-4 Alpaca, Alpaca, OpenAssistant, FLAN) Natural Questions Wizard of Wikipedia FEVER KILT ASQA ARC-Easy OpenBookQA PubHealth ARC-Challenge PopQA TriviaQA-unfiltered
Benchmarks: PopQA TriviaQA-unfiltered PubHealth ARC-Challenge FActScore biographies ALCE-ASQA
Metrics: accuracy FactScore str-em rouge MAUVE citation precision citation recall human S&P
Baselines: Llama2-chat-13B ChatGPT Ret-ChatGPT Ret-Llama2-chat Alpaca Llama2-FT SAIL Toolformer CoVE perplexity.ai
Builds on: retrieval-augmented generation Toolformer SAIL RLHF with PPO Contriever
Closest papers by meaning
- Retrieval is Not Enough: Enhancing RAG through Test-Time Critique and Optimization (NeurIPS 2025 · similarity 0.92)
- Long-Context LLMs Meet RAG: Overcoming Challenges for Long Inputs in RAG (ICLR 2025 · similarity 0.91)
- Speculative RAG: Enhancing Retrieval Augmented Generation through Drafting (ICLR 2025 · similarity 0.91)
- A Theory for Token-Level Harmonization in Retrieval-Augmented Generation (ICLR 2025 · similarity 0.90)
- Inference Scaling for Long-Context Retrieval Augmented Generation (ICLR 2025 · similarity 0.90)
- Improving Retrieval Augmented Language Model with Self-Reasoning (AAAI 2025 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.