Research map / Large language models / Retrieval and knowledge
Tree-of-Reasoning Question Decomposition for Complex Question Answering with Large Language Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes TRQA, a reasoning-tree-based question decomposition framework (T5 tree constructor, prompt- or T5-based question generator, ColBERTv2 retrieval with gpt-3.5-turbo, and bottom-up answer aggregation) for multi-hop QA.
Problem
Existing chain-of-thought with retrieval methods decompose complex questions based on a single compositionality type (nest-type), handling branch-type and mixed-type questions poorly, and retrieve inefficiently for lengthy questions.
Approach
Represent a complex question as a reasoning tree built from dependency parse trees with [NEST] and [BRANCH] separators, generate sub-questions for leaf nodes, retrieve and answer each with an LLM and IR, then aggregate answers bottom-up along the tree.
Key results
- TRQA reaches 61.2 EM on HotpotQA, 35.2 EM on HyBridQA, 24.2 EM on Musique and 52.6 EM on WikiMultiHopQA, outperforming the strongest baseline DSP (53.0, 33.2, 16.2, 43.2 EM).
- TRQA w/o IR (55.4 HotpotQA EM) still beats all CoT-style prompting baselines without retrieval.
- TRQA makes the fewest API interactions per question (2.68 on HyBridQA vs 4.28 for Self-Ask).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use TRQA instead of DSP | multi-hop QA · HotpotQA | EM: 53.0 -> 61.2 | Helped |
| use TRQA instead of DSP | multi-hop QA · HyBridQA | EM: 33.2 -> 35.2 | Helped |
| use TRQA instead of React | multi-hop QA · HyBridQA | Recall: 35.6 -> 42.4 | Helped |
| use TRQA instead of DSP | multi-hop QA · Musique | EM: 16.2 -> 24.2 | Helped |
| use TRQA instead of DSP | multi-hop QA · WMHQA | EM: 43.2 -> 52.6 | Helped |
| add IR to TRQA (ablation) | TRQA · HotpotQA | EM: 55.4 -> 61.2 | Helped |
| add IR to TRQA (ablation) | TRQA · HyBridQA | EM: 28.6 -> 35.2 | Helped |
| add IR to TRQA (ablation) | TRQA · Musique | EM: 18.6 -> 24.2 | Helped |
| add IR to TRQA (ablation) | TRQA · WMHQA | EM: 44.8 -> 52.6 | Helped |
| add AAM (ablation) | TRQA · HotpotQA | Recall: 69.2 -> 62.4 | Hurt |
| add AAM (ablation) | TRQA · HyBridQA | EM: 31.8 -> 35.2 | Helped |
| add AAM (ablation) | TRQA · Musique | EM: 22.4 -> 24.2 | Helped |
Surprising findings
- Removing the IR module on HyBridQA and Musique lowers recall while the w/o AAM variant gets higher recall on HotpotQA than full TRQA in Table 1, suggesting trade-offs between metrics.
Limitations
- Relies on small hand-annotated training sets (about 700 and 400 samples) and inter-annotator agreement of 0.76-0.87.
- Baselines were re-run on gpt-3.5-turbo rather than their original models, and the paper gives limited detail on the setup.
- Paper does not release code or data; reported results include some inconsistent table references.
Experimental setup
Datasets: HotpotQA HyBridQA Musique WikiMultiHopQA RTrees D2Q
Benchmarks: HotpotQA HyBridQA Musique WikiMultiHopQA
Metrics: Cover-EM Recall
Baselines: CoT Auto-CoT Recite-and-answer Self-Ask Least-to-Most Plan-And-Solve React DSP
Builds on: Chain-of-Thought Self-Ask DSP Least-to-Most ReAct
Closest papers by meaning
- Harnessing Large Language Models for Knowledge Graph Question Answering via Adaptive Multi-Aspect Retrieval-Augmentation (AAAI 2025 · similarity 0.89)
- Question Calibration and Multi-Hop Modeling for Temporal Question Answering (AAAI 2024 · similarity 0.88)
- Debate on Graph: A Flexible and Reliable Reasoning Framework for Large Language Models (AAAI 2025 · similarity 0.88)
- Knowledge Editing with Dynamic Knowledge Graphs for Multi-Hop Question Answering (AAAI 2025 · similarity 0.88)
- MEQA: A Benchmark for Multi-hop Event-centric Question Answering with Explanations (NeurIPS 2024 · similarity 0.88)
- Cooperative Retrieval-Augmented Generation for Question Answering: Mutual Information Exchange and Ranking by Contrasting Layers (NeurIPS 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.