Research map / Large language models / Retrieval and knowledge
Retrieval is Accurate Generation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes phrase-retrieval language modeling that generates text by retrieving context-aware phrases from Wikipedia, with training oracles built from linguistic heuristics and iterative self-reinforcement.
Problem
Standard LMs generate by selecting tokens from a fixed vocabulary, and existing retrieval-augmented LMs still use two-stage document-then-phrase pipelines; constructing training oracles that segment text into retrievable phrases from many possible documents is hard.
Approach
Replace token-level next-token prediction with next-phrase retrieval over a context-aware phrase table (no document-retrieval stage), where training oracles are initialized by syntactic constituents, BM25 and phrase-encoder similarity, then refined by iteratively letting the model re-select its own top-k targets.
Key results
- On OpenbookQA accuracy rises from 23.47% (Base LM) to 36.27% for the phrase-retrieval model, and it is the best among the compared retrieval baselines (CoG 35.47%).
- MAUVE on open-ended generation improves from 42.61 (Base LM) to 81.58, versus 52.27 for CoG.
- Latency is 1.29x the base LM, the lowest among retrieval-augmented baselines (CoG 4.40x, kNN-LM 6.29x, RETRO 1.51x).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use context-aware phrase retrieval instead of standalone-token generation (ablation) | Ours · OpenbookQA test | accuracy: 23.73 -> 36.27 | Helped |
| Use context-aware phrase retrieval instead of standalone-token generation (ablation) | Ours · TruthfulQA test | accuracy: 28.63 -> 34.27 | Helped |
| Use context-aware phrase retrieval rather than token generation | GPT-2 base LM · OpenbookQA test | accuracy: 23.47% to 36.27% | Helped |
| Expand the phrase index without retraining | Ours · TruthfulQA test | accuracy: 34.27 -> 39.59 | Helped |
| Replace the general phrase index with a domain-specific medical index | Ours · MedMCQA validation | accuracy: 29.44 -> 29.50 | Helped |
| Replace the general phrase index with a domain-specific medical index | Ours · Med-USMILE test | accuracy: 25.69 -> 26.38 | Helped |
| Add self-reinforcement, progressing from no SR through successive rounds (ablation) | Ours · MiniPile test open-ended generation | MAUVE: 7.86 -> 64.49 -> 81.58 | Helped |
| Add self-reinforcement (ablation) | Ours · Knowledge-intensive tasks | accuracy: w/o SR: 34.11, 37.07, 27.14, 30.32, 25.85; round2: 34.27, 36.27, 28.24, 29.44, 25.69 | No clear effect |
| Vary the number of retrieved candidates k (ablation) | Ours · Knowledge-intensive tasks | Avg. accuracy: 30.62–30.83 | No clear effect |
| Use phrase retrieval rather than standalone tokens | Ours · MiniPile test open-ended generation | MAUVE: 42.61% to 81.58% | Helped |
| Use Ours rather than CoG | Ours · MiniPile test open-ended generation | MAUVE: 52.27 -> 81.58 | Helped |
| Use Ours rather than kNN-LM | Ours · MiniPile test open-ended generation | Latency: 6.29x -> 1.29x | Helped |
Surprising findings
- Self-reinforcement has little effect on knowledge-intensive QA accuracy but is essential for open-ended MAUVE (w/o SR MAUVE 7.86).
- Base LM fine-tuned on Wikipedia frequently outputs 'References' and 'External Links' text, which largely explains its lower MAUVE relative to the non-fine-tuned model.
Limitations
- Authors describe the result as a proof-of-concept; scalability to larger corpora with far more phrases is unresolved.
- Model is not aligned to human instructions; alignment is left to future work.
- Phrase-level factuality is not directly measured, and MAUVE alone may not capture generation quality, as the authors note.
Experimental setup
Datasets: MiniPile English Wikipedia (March 1, 2022) Wikipedia medical terms
Benchmarks: OpenbookQA ARC-Challenge TruthfulQA MedMCQA Med-USMLE (Med-USMILE) MiniPile test (open-ended generation)
Metrics: accuracy MAUVE coherence diversity latency human Likert ratings
Baselines: Base LM kNN-LM RETRO CoG
Builds on: CoG DensePhrases GPT-2 kNN-LM RETRO
Closest papers by meaning
- Towards Verifiable Text Generation with Generative Agent (AAAI 2025 · similarity 0.89)
- A Theory for Token-Level Harmonization in Retrieval-Augmented Generation (ICLR 2025 · similarity 0.89)
- Provence: efficient and robust context pruning for retrieval-augmented generation (ICLR 2025 · similarity 0.88)
- A Systematic Exploration of Knowledge Graph Alignment with Large Language Models in Retrieval Augmented Generation (AAAI 2025 · similarity 0.88)
- DynamicRAG: Leveraging Outputs of Large Language Model as Feedback for Dynamic Reranking in Retrieval-Augmented Generation (NeurIPS 2025 · similarity 0.87)
- UniGen: A Unified Generative Framework for Retrieval and Question Answering with Large Language Models (AAAI 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.