Research map / Vision-language and multimodal / Multimodal LLMs

CMMCoT: Enhancing Complex Multi-Image Comprehension via Multi-Modal Chain-of-Thought and Memory Augmentation

AAAI 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Chain-of-thought multimodal reasoning · 1 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes CMMCoT, an interleaved multimodal chain-of-thought framework with coordinate-grounded entity image tokens and a test-time memory retrieval module (RIFREM), fine-tuned on a new 260k multi-image slow-thinking dataset built on Qwen2.5-VL-7B.

Problem

Multimodal slow-thinking methods work for single images but struggle on complex multi-image comprehension, since they rely mainly on text-based intermediate reasoning and lack cross-image visual tracking and memory.

Approach

Interleave reasoning text with bounding-box coordinates and extracted entity image tokens as supervision during SFT, and at inference retrieve cached per-layer key/value pairs of multi-image inputs via cross-attention (RIFREM) to enrich entity query features without extra parameters.

Key results

• Qwen2.5-VL-7B with CMMCoT reaches 74.6 average on five multi-image benchmarks vs 73.2 for the base model.
• The 3B variant reaches 71.4 average on multi-image benchmarks vs 68.8 for Qwen2.5-VL-3B.
• Grounding plus entity images raises the multi-image average from 73.2 to 74.4; adding RIFREM at inference reaches 74.6.

Surprising findings

• Fine-tuning LLaVA-v1.5, LLaVA-v1.6, Mantis-Idefics2, and LLaVA-OV on CMMCoT-260K degrades their performance, which the authors attribute to lacking localization ability.

Limitations

• RIFREM introduces non-trivial latency overhead that grows with the number of layers where it is applied.
• Gains depend on the base model having native localization ability; fine-tuning models without grounding degrades performance.

Experimental setup

Datasets: GRIT Flickr30k-Entities VoCoT MANTIS NLVR2 ImageCoDe Birds-to-Words DreamSim

Benchmarks: BLINK Mantis-Eval NLVR2 MVBench Q-Bench MMMU MMStar SQA RealWorldQA MME POPE HallBench

Metrics: accuracy average score

Baselines: LLaVA-v1.5 LLaVA-v1.6 LLaVA-v1.5-MIA-DPO LLaVA-OV Mantis-Idefics2 Qwen2-VL InternVL3 Qwen2.5-VL GPT-4V GPT-4o

Builds on: Qwen2-VL MVoT VoCoT Mantis

Closest papers by meaning

• Mimic Human Cognition, Master Multi-Image Reasoning: A Meta-Action Framework for Enhanced Visual Understanding (CVPR 2026 · similarity 0.91)
• Visual Thoughts: A Unified Perspective of Understanding Multimodal Chain-of-Thought (NeurIPS 2025 · similarity 0.90)
• VGR: Visual Grounded Reasoning (ICLR 2026 · similarity 0.90)
• Imagine While Reasoning in Space: Multimodal Visualization-of-Thought (ICML 2025 · similarity 0.90)
• Let’s Think with Images Efficiently! An Interleaved-Modal Chain-of-Thought Reasoning Framework with Dynamic and Precise Visual Thoughts (AAAI 2026 · similarity 0.90)
• Machine Mental Imagery: Empower Multimodal Reasoning with Latent Visual Tokens (CVPR 2026 · similarity 0.90)