Research map / Large language models / Retrieval and knowledge
RAPTOR: Recursive Abstractive Processing for Tree-Organized Retrieval
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes RAPTOR, which builds a tree by recursively clustering (GMM+UMAP) and summarizing text chunks and retrieves across tree layers via a collapsed-tree search, improving QA on NarrativeQA, QASPER, and QuALITY.
Problem
Retrieval-augmented LMs retrieve only short contiguous chunks, which limits holistic understanding of long documents needed for thematic, multi-hop questions.
Approach
Recursively embed, cluster (soft GMM clustering with UMAP and BIC-selected cluster count), and summarize chunks into a bottom-up tree, then retrieve from all layers at once (collapsed tree) so that answers can draw on both granular details and high-level summaries.
Key results
- RAPTOR with GPT-4 reaches 82.6% accuracy on QuALITY test (76.2% on hard subset), up from prior best 62.3%.
- RAPTOR + GPT-4 achieves 55.7 F1 Match on QASPER, above CoLT5 XL's 53.9.
- With UnifiedQA-3B, RAPTOR + SBERT improves QuALITY accuracy from 54.9% to 56.6% over the same retriever without the tree.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add RAPTOR recursive clustering and summarization to SBERT retrieval | UnifiedQA-3B · QuALITY | accuracy: 54.9% -> 56.6% | Helped |
| add RAPTOR recursive clustering and summarization to SBERT retrieval | UnifiedQA-3B · QASPER | Answer F1: 36.23% -> 36.70% | Helped |
| add RAPTOR recursive clustering and summarization to SBERT retrieval | UnifiedQA-3B · NarrativeQA | ROUGE: 29.26% -> 30.87% | Helped |
| add RAPTOR recursive clustering and summarization to SBERT retrieval | UnifiedQA-3B · NarrativeQA | BLEU-1: 22.56% -> 23.50% | Helped |
| use RAPTOR retrieval instead of DPR | GPT-4 · QASPER | F-1 Match: 53.0 -> 55.7 | Helped |
| use RAPTOR retrieval instead of DPR | GPT-3 · QuALITY dev | accuracy: 60.4 -> 62.4 | Helped |
| use RAPTOR retrieval instead of CoLISA | GPT-4 · QuALITY test set | accuracy: 62.3 -> 82.6 | Helped |
| use RAPTOR + UnifiedQA instead of Retriever + Reader | NarrativeQA · NarrativeQA | METEOR: 11.1 -> 19.1 | Helped |
| replace semantic clustering with contiguous-chunk balanced-tree construction (ablation) | RAPTOR + SBERT embeddings + UnifiedQA · QuALITY | accuracy: 56.6% -> 55.8% | Hurt |
| query all three layers starting at Layer 2 instead of only Layer 0 (ablation) | RAPTOR · QuALITY Story 1 | accuracy: 57.9 -> 73.68 | Helped |
| query two layers starting at Layer 1 instead of one layer at Layer 1 (ablation) | RAPTOR · QuALITY Story 2 | accuracy: 47.1 -> 64.7 | Helped |
| query all three layers starting at Layer 2 instead of only Layer 2 (ablation) | RAPTOR · QuALITY Story 3 | accuracy: 61.1 -> 83.3 | Helped |
Surprising findings
- Recency-based balanced tree summarization of contiguous chunks underperforms clustering only slightly (55.8% vs 56.6%), but clustering still helps.
- Between 18.5% and 57% of retrieved nodes come from non-leaf (summary) layers.
Limitations
- About 4% of sampled summaries contained minor hallucinations, assessed by manual annotation of 150 nodes.
- Collapsed-tree retrieval requires similarity search over all nodes, though FAISS can help.
- Reliance on GPT-3.5-turbo for summarization and GPT-4 for best results incurs API cost.
Experimental setup
Datasets: NarrativeQA QASPER QuALITY
Benchmarks: NarrativeQA QASPER QuALITY
Metrics: accuracy F1 Match ROUGE-L BLEU-1 BLEU-4 METEOR Answer F1
Baselines: SBERT BM25 DPR Title + Abstract LongT5 XL CoLT5 XL CoLISA Recursively Summarizing Books Retriever + Reader BiDAF Longformer-base DPR and DeBERTaV3-large
Builds on: retrieval-augmented language models UMAP SBERT Dense Passage Retrieval recursive summarization
Closest papers by meaning
- Retriever-Free Retrieval-Augmented Reasoning via Corpus-Traversing MCTS (NeurIPS 2026 · similarity 0.87)
- Autofocus Retrieval: An Effective Pipeline for Multi-Hop Question Answering With Semi-Structured Knowledge (NeurIPS 2026 · similarity 0.87)
- Cooperative Retrieval-Augmented Generation for Question Answering: Mutual Information Exchange and Ranking by Contrasting Layers (NeurIPS 2025 · similarity 0.86)
- RECOMP: Improving Retrieval-Augmented LMs with Context Compression and Selective Augmentation (ICLR 2024 · similarity 0.86)
- SiReRAG: Indexing Similar and Related Information for Multihop Reasoning (ICLR 2025 · similarity 0.86)
- Simple is Effective: The Roles of Graphs and Large Language Models in Knowledge-Graph-Based Retrieval-Augmented Generation (ICLR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.