Research map / Large language models / Reasoning and chain of thought
MindMap: Constructing Evidence Chains for Multi-Step Reasoning in Large Language Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes MindMap, which groups facts into subject-centric evidence chains, summarizes them with an LLM, and plugs these into CoT and Selection-Inference to improve multi-step logical reasoning on bAbI and ProofWriterOWA.
Problem
LLMs struggle with multi-step logical reasoning over a set of facts and rules, and existing step-by-step methods treat each fact as isolated evidence, ignoring relations among facts.
Approach
Organize available facts into evidence chains, one per subject extracted with Stanford CoreNLP, summarize each chain with an LLM in a few-shot setting, and let the reasoning framework (CoT or SI) select and use chains instead of individual facts.
Key results
- On bAbI task-3, CoT+MindMap reaches 0.340 accuracy vs 0.281 for CoT (21.0% relative gain).
- On ProofWriter-PD depth-0, CoT+MindMap reaches 0.711 vs 0.545 for CoT (30.5% relative gain).
- MindMap covers 69% and 47% of supporting facts on bAbI task-2 and task-3 vs 48% and 23% for CoT.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add MindMap to CoT | CoT with Vicuna-13B · ProofWriter-PD depth-0 | prediction accuracy: 0.545 -> 0.711 | Helped |
| Add MindMap to CoT | CoT with Vicuna-13B · ProofWriter-PD depth-1 | prediction accuracy: 0.559 -> 0.700 | Helped |
| Add MindMap to CoT | CoT with Vicuna-13B · ProofWriter-PD depth-2 | prediction accuracy: 0.552 -> 0.683 | Helped |
| Add MindMap to CoT | CoT with Vicuna-13B · ProofWriter-PD depth-5 | prediction accuracy: 0.540 -> 0.589 | Helped |
| Add MindMap to CoT | CoT with Vicuna-13B · ProofWriter-PUD depth-0 | prediction accuracy: 0.417 -> 0.469 | Helped |
| Add MindMap to CoT | CoT with Vicuna-13B · ProofWriter-PUD depth-5 | prediction accuracy: 0.419 -> 0.441 | Helped |
| Add MindMap to CoT | CoT with Vicuna-13B · bAbI task-1 | prediction accuracy: 0.607 -> 0.881 | Helped |
| Add MindMap to CoT | CoT with Vicuna-13B · bAbI task-2 | prediction accuracy: 0.467 -> 0.473 | Helped |
| Add MindMap to CoT | CoT with Vicuna-13B · bAbI task-3 | prediction accuracy: 0.281 -> 0.340 | Helped |
| Use subject-centric evidence chains without evidence chain summarization (ablation) | CoT on bAbI · bAbI task-1 | prediction accuracy: 0.607 -> 0.789 | Helped |
| Use subject-centric evidence chains without evidence chain summarization (ablation) | CoT on bAbI · bAbI task-3 | prediction accuracy: 0.281 -> 0.26 | No clear effect |
| Automatically construct and summarize evidence chains via instruction tuning (ablation) | CoT on bAbI · bAbI task-1 | prediction accuracy: 0.607 -> 0.372 | Hurt |
Surprising findings
- Auto-MindMap, where the model builds and summarizes evidence chains via instruction tuning, substantially decreases performance (e.g., 0.372 on task-1 vs 0.607 for CoT).
- On ProofWriter-PUD, the standard prompting baseline outperforms CoT and SI with or without MindMap, attributed to label imbalance toward Unknown.
Limitations
- Evaluated only with a 13B model (Vicuna-13B) due to compute constraints.
- Subject-centric chain construction is heuristic and may fail with pronoun resolution in practical text.
- Evaluation limited to synthetic datasets; ProofWriterOWA uses only first 1,000 test samples.
Experimental setup
Datasets: bAbI ProofWriterOWA
Benchmarks: bAbI tasks 1-3 ProofWriterOWA (PD and PUD subsets)
Metrics: accuracy relative improvement supporting fact coverage
Baselines: Standard few-shot prompting CoT SI Auto-MindMap
Builds on: chain-of-thought Selection-Inference narrative event chains
Closest papers by meaning
- RATT: A Thought Structure for Coherent and Correct LLM Reasoning (AAAI 2025 · similarity 0.89)
- Understanding Reasoning Ability of Language Models From the Perspective of Reasoning Paths Aggregation (ICML 2024 · similarity 0.89)
- Unlocking the Capabilities of Thought: A Reasoning Boundary Framework to Quantify and Optimize Chain-of-Thought (NeurIPS 2024 · similarity 0.89)
- Training Language Models to Reason Efficiently (NeurIPS 2025 · similarity 0.88)
- Enhancing Chain of Thought Prompting in Large Language Models via Reasoning Patterns (AAAI 2025 · similarity 0.88)
- A State-Transition Framework for Efficient LLM Reasoning (ICLR 2026 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.