Research map / Large language models / Retrieval and knowledge
SiReRAG: Indexing Similar and Related Information for Multihop Reasoning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SiReRAG, a RAG indexing method that builds a similarity tree (RAPTOR-style recursive summaries) and a relatedness tree (entity-linked proposition aggregates with recursive summaries) and flattens both into one retrieval pool for multihop QA.
Problem
Existing RAG indexing methods organize data by either semantic similarity (RAPTOR) or relatedness via entities (HippoRAG, GraphRAG), which leads to insufficient knowledge synthesis for multihop reasoning questions.
Approach
Index both similarity and relatedness explicitly: a similarity tree from GMM-clustered recursive summaries plus a relatedness tree built from entity-grouped propositions aggregated and recursively summarized, with both trees flattened into one unified retrieval pool.
Key results
- SiReRAG (GPT-4o) averages 65.83 F1 across MuSiQue, 2Wiki, and HotpotQA versus 61.21 for RAPTOR (GPT-4o) and 64.03 for HippoRAG (GPT-3.5-Turbo).
- Adding SiReRAG's retrieval pool improves ColBERTv2 average F1 from 59.90 to 63.93 and BM25 from 55.26 to 63.09.
- SiReRAG's TPER values range from 0.499 to 0.600, below 1, with average TPQ 2.315s versus 1.500s for RAPTOR.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| index both similar and related information in a unified retrieval pool | RAPTOR-style indexing on multihop QA · Average across MuSiQue, 2WikiMultiHopQA, and HotpotQA | F1: 61.21 -> 65.83 | Helped |
| remove recursive summaries from the relatedness tree (ablation) | SiReRAG · Average across MuSiQue, 2WikiMultiHopQA, and HotpotQA | F1: 65.83 -> 64.04 | Hurt |
| add individual propositions to the retrieval pool while retaining proposition aggregates and summaries (ablation) | SiReRAG · Average across MuSiQue, 2WikiMultiHopQA, and HotpotQA | F1: 65.83 -> 64.20 | Hurt |
| remove proposition aggregates (ablation) | SiReRAG with individual propositions · Average across MuSiQue, 2WikiMultiHopQA, and HotpotQA | F1: 64.20 -> 60.47 | Hurt |
| remove proposition aggregates and relatedness-tree summaries (ablation) | SiReRAG with individual propositions · Average across MuSiQue, 2WikiMultiHopQA, and HotpotQA | F1: 64.20 -> 58.80 | Hurt |
| replace the proposition-aggregate relatedness tree with entity clustering on chunks (ablation) | SiReRAG · Average across MuSiQue, 2WikiMultiHopQA, and HotpotQA | F1: 65.83 -> 59.70 | Hurt |
| use cross-tree interaction rather than keeping similarity and relatedness trees separate (ablation) | SiReRAG · Average across MuSiQue, 2WikiMultiHopQA, and HotpotQA | F1: 65.83 -> 64.67 | Hurt |
| organize text chunks at different tree levels by abstractiveness | RAPTOR on QuALITY validation · QuALITY validation | Average Accuracy: 78.88 -> 78.76 | No clear effect |
| use GPT-4o instead of GPT-3.5-Turbo for indexing summaries | SiReRAG · Multihop QA benchmarks | F1 | No clear effect |
Surprising findings
- Adding all individual propositions to the retrieval pool hurt QA performance (variant B vs. full SiReRAG), suggesting redundant information degrades results.
- GraphRAG performed worst on multihop QA, often answering 'I don't know'.
Limitations
- Relies on GPT-4o and API-based summarization; indexing cost and efficiency are meaningfully higher than RAPTOR (pool roughly 3x larger).
- Gains over HippoRAG are not uniform; HippoRAG is stronger on 2Wiki.
- Evaluation limited to QA metrics without direct retrieval scores because retrieval pools differ across methods.
Experimental setup
Datasets: MuSiQue 2WikiMultiHopQA HotpotQA BAAI/IndustryCorpus QuALITY MultiHop-RAG ASQA
Benchmarks: MuSiQue 2WikiMultiHopQA HotpotQA Single-hop MuSiQue questions MultiHop-RAG ASQA
Metrics: EM F1 TPQ TPER STR-EM Disambig-F1 Disambiguation-Rouge
Baselines: RAPTOR HippoRAG GraphRAG BM25 ColBERTv2 self-ask
Builds on: RAPTOR HippoRAG GraphRAG Distill-SynthKG ColBERTv2
Closest papers by meaning
- Simple is Effective: The Roles of Graphs and Large Language Models in Knowledge-Graph-Based Retrieval-Augmented Generation (ICLR 2025 · similarity 0.88)
- Improving Consistency in Retrieval Augmented Systems With Group Similarity Rewards (NeurIPS 2026 · similarity 0.88)
- From Complex to Atomic: Enhancing Augmented Generation via Knowledge-Aware Dual Rewriting and Reasoning (ICML 2025 · similarity 0.88)
- HyperGraphRAG: Retrieval-Augmented Generation via Hypergraph-Structured Knowledge Representation (NeurIPS 2025 · similarity 0.88)
- Cooperative Retrieval-Augmented Generation for Question Answering: Mutual Information Exchange and Ranking by Contrasting Layers (NeurIPS 2025 · similarity 0.88)
- Retrieval is Not Enough: Enhancing RAG through Test-Time Critique and Optimization (NeurIPS 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.