Research map / Vision-language and multimodal / Multimodal LLMs
Chain-of-thought multimodal reasoning
Improves multimodal LLM reasoning with latent thinking, long chains, and structured scene-graph reasoning.
332 accepted papers at ICML, NeurIPS, ICLR, CVPR and AAAI (2016–2026) · an approach in Multimodal LLMs, keyword cluster “vlms · visual reasoning · chain”
State History Tried Baselines Checklist Preprints Who Export
Where it stands
Not enough benchmark data Fewer than four papers report the same benchmark, so there is no record to follow.
17 papers in 2023–24 and 312 in 2025–26 (at ICML, NeurIPS, ICLR, CVPR and AAAI); its share of the topic grew, from 8.2% to 14.5%.
Open questions the data points to
- Papers report Qwen2.5-VL on MME (score) anywhere from 1660 to 2347: comparisons against it depend on whose number you take.
- Papers report Qwen2.5-VL on HallusionBench (score) anywhere from 49 to 61.3: comparisons against it depend on whose number you take.
How this is decided
A benchmark is tracked when at least four papers of this approach report their own method on the same dataset and metric in at least two different years. Its record is the best number reported up to 2024 compared with the best reported in 2025–26: still improving means at least 1% better, flat means at most 0.2% better with at least two papers from 2025–26 trying. Numbers on a different scale from the rest (a fraction among percentages) and far outliers are left out. Results come from different papers with different setups, so they are signals, not head-to-head comparisons. The share of papers is counted at ICML, NeurIPS, ICLR, CVPR and AAAI.
How it got here
Most cited paper of each year
- 2016 · A Semantical Analysis of Second-Order Propositional Modal Logic (AAAI 2016 · 11 citations)
- 2021 · Transformation Driven Visual Reasoning (CVPR 2021 · 18 citations)
- 2023 · GAMR: A Guided Attention Model for (visual) Reasoning (ICLR 2023 · 4 citations)
- 2024 · Compositional Chain-of-Thought Prompting for Large Multimodal Models (CVPR 2024 · 92 citations)
- 2025 · Insight-V: Exploring Long-Chain Visual Reasoning with Multimodal Large Language Models (CVPR 2025 · 13 citations)
- 2026 · MedAgent-Pro: Towards Evidence-based Multi-modal Medical Diagnosis via Reasoning Agentic Workflow (ICLR 2026 · 13 citations)
Earliest papers here
- A Semantical Analysis of Second-Order Propositional Modal Logic (AAAI 2016 · 11 citations)
- Transformation Driven Visual Reasoning (CVPR 2021 · 18 citations)
- ReaSCAN: Compositional Reasoning in Language Grounding (NeurIPS 2021 · 5 citations)
What has been tried
76 papers here report 869 measured changes (a component added, removed or swapped, compared with a baseline): helped 68%, no clear effect 5%, hurt 24%, mixed 3%. Papers mostly publish changes that helped, so "helped" is over-represented.
Ablations in recent papers
| What was changed | Measured on | Result | Effect | Paper |
|---|---|---|---|---|
| Require object bounding-box locations in the scene graph | SEEDBench-Image · weighted average | 72.1 -> 70.5 | Hurt | Compositional Chain-of-Thought Prompting for Large Multimodal Models (CVPR 2024 · 92 citations) |
| Remove attend-describe components | A-OKVQA val · accuracy | 46.41 -> 43.48 | Hurt | Visual Chain-of-Thought Prompting for Knowledge-Based Visual Reasoning (AAAI 2024 · 43 citations) |
| Distill verified visual programs as synthesized CoT instead of training without synthesized CoT | GQA · EM | +1.6 | Helped | Visual Program Distillation: Distilling Tools and Programmatic Reasoning into Vision-Language Models (CVPR 2024 · 34 citations) |
| remove the reasoning agent | Visual reasoning ablation benchmarks · Average | 62.1 -> 59.8 | Hurt | Insight-V: Exploring Long-Chain Visual Reasoning with Multimodal Large Language Models (CVPR 2025 · 13 citations) |
| Add ADS to generate interleaved-modal rather than text-only rationales | M^3CoT 1-shot · accuracy | 29.2 -> 32.3 | Helped | Interleaved-Modal Chain-of-Thought (CVPR 2025 · 11 citations) |
| Append “Let's think step by step:” to questions | C-VQA-Real numerical direct · accuracy | 63.2 -> 34.2 (-29.0) | Hurt | What If the TV Was Off? Examining Counterfactual Reasoning Abilities of Multi-modal Language Models (CVPR 2024 · 10 citations) |
| Include point-location identification before depth-map reasoning | HardBLINK, 3 points · accuracy (%) | 56.4 -> 66.9 | Helped | Perception Tokens Enhance Visual Reasoning in Multimodal Language Models (CVPR 2025 · 9 citations) |
| removing Stable Diffusion pre-trained initialization (random VAE and UNet) | ScienceQA test · Avg accuracy (%) | -4.88 | Hurt | Multi-Modal Latent Space Learning for Chain-of-Thought Reasoning in Language Models (AAAI 2024 · 9 citations) |
| Increase input frames from 8 to 32 | ECBench · Overall mean | 45.96 -> 50.35 | Helped | ECBench: Can Multi-modal Foundation Models Understand the Egocentric World? A Holistic Embodied Cognition Benchmark (CVPR 2025 · 7 citations) |
| apply iterative SFT→RL rather than RL-only | HallusionBench · accuracy | 66.8 -> 70.2 | Helped | OpenVLThinker: Complex Vision-Language Reasoning via Iterative SFT-RL Cycles (NeurIPS 2025 · 6 citations) |
| Apply three-step iterative perception instead of single-step perception | LongVideoBench · accuracy | 61.4 -> 62.6 | Helped | VideoChat-R1.5: Visual Test-Time Scaling to Reinforce Multimodal Reasoning by Iterative Perception (NeurIPS 2025 · 6 citations) |
| add proposal adapter | GQA · accuracy | 62.1 -> 63.2 | Helped | Visual Agents as Fast and Slow Thinkers (ICLR 2025 · 4 citations) |
Baseline numbers that disagree
The same baseline on the same benchmark and metric, reported more than 5% apart by different papers here. Training budgets, data splits and reimplementations differ, so state which number you compare with.
| Baseline | Benchmark | Metric | Reported | Papers |
|---|---|---|---|---|
| Qwen2.5-VL | MME | score | 1660 – 2347 | 2 papers, e.g. NeurIPS 2025 and NeurIPS 2025 |
| Qwen2.5-VL | HallusionBench | score | 49 – 61.3 | 2 papers, e.g. NeurIPS 2025 and NeurIPS 2025 |
What reviewers will expect
Datasets and baselines named in the 312 papers of this approach from 2025–2026 that have a full-text summary. Leaving out the common ones is a frequent review question.
Datasets
- MathVista · 22 of 312 papers
- ScienceQA · 17 of 312 papers
- MathVision · 17 of 312 papers
- ChartQA · 17 of 312 papers
- GQA · 16 of 312 papers
- RefCOCO · 14 of 312 papers
- RefCOCOg · 14 of 312 papers
- M3CoT · 12 of 312 papers
Baselines
- GPT-4o · 105 of 312 papers
- Qwen2.5-VL-7B · 47 of 312 papers
- Gemini-2.5-Pro · 45 of 312 papers
- Gemini-2.5-Flash · 25 of 312 papers
- GPT-5 · 22 of 312 papers
- Gemini-2.0-Flash · 21 of 312 papers
- Qwen2.5-VL · 21 of 312 papers
- Qwen2.5-VL-72B · 21 of 312 papers
Recent preprints
Loading recent arXiv preprints…
Who publishes on it
Institutions on the most papers of this approach since 2023 (of 329 papers; from OpenAlex and the papers' affiliations).
- Tsinghua University · 13 papers
- Alibaba Group · 12 papers
- The Chinese University of Hong Kong · 11 papers
- Microsoft · 9 papers
- Peking University · 9 papers
- Zhejiang University · 7 papers
- Tencent · 7 papers
- University of Science and Technology of China · 7 papers
- Shanghai Jiao Tong University · 7 papers
- University of California, Los Angeles · 6 papers
Use it
Download BibTeX of 24 key papersOpen the topic in the interactive map
Get alerts. Describe your idea in scime atlas and save it (free scime account): you get a weekly email when new preprints land close to it.
Related approaches
Same keyword cluster
- VLM capability analysis (264 papers)
- Multimodal LLM benchmarks (252 papers)
- Spatial reasoning in VLMs (207 papers)
- Reinforcement learning for visual reasoning (163 papers)
Elsewhere in the topic
- Instruction-tuned multimodal assistants (199 papers)
- Benchmarks and analysis of MLLMs (181 papers)
- Interleaved image-text generation with MLLMs (153 papers)
- Unified any-modality language models (152 papers)
- Attention-based visual token pruning (117 papers)
- Decoding-time hallucination mitigation (91 papers)
- Training-free token reduction for MLLMs (90 papers)
- Visual instruction tuning (68 papers)
Benchmark numbers, ablations and baseline numbers are extracted by AI from each paper's full text and can be wrong; the paper is the authority. Data: results index of 2026-11-02, citation counts from OpenAlex (2026-10-09).