Research map / Large language models / Retrieval and knowledge
Sufficient Context: A New Lens on Retrieval Augmented Generation Systems
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Defines sufficient context for RAG instances, builds an LLM-based sufficient-context autorater, uses it to stratify model errors across RAG datasets, and proposes a selective generation method combining sufficiency and self-confidence to reduce hallucinations.
Problem
RAG errors are hard to attribute: it is unclear whether LLMs fail to use retrieved context or whether the context is insufficient to answer the query. Prior relevance notions are not anchored to a precise definition of whether context suffices.
Approach
Define sufficient context as a property of (query, context) pairs requiring only that some plausible answer exists, without needing a ground truth answer; use an LLM autorater (Gemini 1.5 Pro 1-shot) to label instances and stratify model responses, then combine the sufficiency label with self-rated confidence in a logistic regression for selective generation.
Key results
- Gemini 1.5 Pro (1-shot) sufficient context autorater reaches 93% accuracy and 0.935 F1 on a 115-instance human-labeled set, beating FLAMe, TRUE-NLI and Contains GT.
- Models output correct answers 35-62% of the time with insufficient context, and hallucinate more than they abstain with sufficient context.
- Sufficiency-plus-confidence selective generation improves selective accuracy by over 10% for Gemma 27B on HotPotQA at high accuracy regions and over 5% for Gemini 1.5 Pro near 70% coverage.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use the sufficient-context signal with model confidence for selective generation instead of model confidence alone | Gemma 27B and Gemini 1.5 Pro · HotPotQA | selective accuracy: over 10% for Gemma 27B in the highest accuracy regions; over 5% for Gemini 1.5 Pro near the 70% coverage region | Helped |
| Combine sufficient-context information with confidence for selective generation | Gemini, GPT, and Gemma · RAG queries answered by the model | fraction of correct answers among responses: 2--10% | Helped |
| Add retrieved context (RAG) rather than answer closed-book | Claude 3.5 Sonnet · QA queries | abstention fraction: 84.1% to 52% | Hurt |
| Add retrieved context (RAG) rather than answer closed-book | GPT 4o · QA queries | abstention fraction: 34.4% to 31.2% | Hurt |
| Add retrieved context (RAG) rather than answer closed-book | Gemini 1.5 Pro · QA queries | abstention fraction: 100% to 18.6% | Hurt |
| Fine-tune on ground-truth answers rather than Vanilla RAG | Mistral 3 7B Instruct · Musique | % Correct: 28.8 -> 31.4 | Helped |
| Change 20% of random training answers to “I don't know” instead of keeping ground-truth answers (ablation) | Mistral 3 7B Instruct · Musique | % Correct: 31.4 -> 23 | Hurt |
| Change 20% of random training answers to “I don't know” instead of keeping ground-truth answers (ablation) | Mistral 3 7B Instruct · HotPotQA | % Correct: 43.4 -> 41.6 | Hurt |
| Change 20% of insufficient-context training answers to “I don't know” instead of keeping ground-truth answers (ablation) | Mistral 3 7B Instruct · Musique | % Correct: 31.4 -> 23 | Hurt |
| Change 20% of insufficient-context training answers to “I don't know” instead of keeping ground-truth answers (ablation) | Mistral 3 7B Instruct · HotPotQA | % Correct: 43.4 -> 41.2 | Hurt |
| Add “I don't know” answers to training data (Data Mix 2 or 3) instead of using ground-truth answers only (ablation) | Mistral 3 7B Instruct · Musique and HotPotQA | % Abstain: 0 -> 1.2; 0 -> 2.2; 0 -> 0.8; 0 -> 2 | Helped |
| Add sufficient-context signal to confidence-based selective generation | Gemma 27B · Musique | selective accuracy | No clear effect |
Surprising findings
- Models answer correctly in many cases even when context is insufficient, and this persists after filtering out questions answered correctly closed-book.
- With RAG, models abstain less than without RAG (e.g., Gemini 1.5 Pro abstention drops from 100% to 18.6% on FreshQA).
Limitations
- Analysis focuses on QA datasets; summarization and other tasks not studied.
- Does not explore how different retrieval methods affect sufficient context rates.
- Autorater evaluated on a small 115-instance human-labeled set; fine-tuning results were unstable across checkpoints.
Experimental setup
Datasets: FreshQA Musique-Ans HotPotQA PopQA Natural Questions EntityQuestions
Benchmarks: FreshQA Musique-Ans HotPotQA
Metrics: accuracy F1 score precision recall % correct % abstain % hallucinate selective accuracy vs coverage
Baselines: FLAMe TRUE-NLI Contains GT Gemini 1.5 Pro (0-shot) P(True) self-confidence P(Correct) self-confidence Vanilla RAG closed-book
Builds on: AIS attributable to identified sources REPLUG FlashRAG FLAMe TRUE NLI
Closest papers by meaning
- Influence Guided Context Selection for Effective Retrieval-Augmented Generation (NeurIPS 2025 · similarity 0.90)
- Long-Context LLMs Meet RAG: Overcoming Challenges for Long Inputs in RAG (ICLR 2025 · similarity 0.89)
- Provence: efficient and robust context pruning for retrieval-augmented generation (ICLR 2025 · similarity 0.89)
- Inference Scaling for Long-Context Retrieval Augmented Generation (ICLR 2025 · similarity 0.89)
- RankRAG: Unifying Context Ranking with Retrieval-Augmented Generation in LLMs (NeurIPS 2024 · similarity 0.88)
- LaRA: Benchmarking Retrieval-Augmented Generation and Long-Context LLMs – No Silver Bullet for LC or RAG Routing (ICML 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.