Research map / Vision-language and multimodal / Multimodal LLMs

Let’s Think with Images Efficiently! An Interleaved-Modal Chain-of-Thought Reasoning Framework with Dynamic and Precise Visual Thoughts

AAAI 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Chain-of-thought multimodal reasoning · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes DaP-ICoT, an interleaved-modal chain-of-thought framework that inserts object-level visual thoughts only when MLLM confidence (mean top-1/top-2 logit margin) is low, selected via SAM2 segmentation and cross-modal attention relevance.

Problem

Existing interleaved-modal chain-of-thought (ICoT) methods insert visual thoughts at fixed steps and use broken, discontinuous image tokens, causing inefficient and semantically incoherent multimodal reasoning.

Approach

Dynamic Visual Thought Integration gates image insertion by a logit-margin confidence score per reasoning step, and Precise Visual Thought Guidance replaces patch tokens with SAM2 object-level sub-images chosen by cross-modal attention relevance to the rationale.

Key results

• On M3CoT 0-shot with Chameleon-7B, DaP-ICoT reaches 41.0% accuracy vs 31.0% for the second-best (SCAFFOLD).
• With Qwen2-VL-7B on M3CoT, DaP-ICoT reaches 57.2% (0-shot) and 58.7% (1-shot) accuracy, and uses 314 tokens on average vs 1,146 for ICoT (72.6% reduction).
• DaP-ICoT inserts 1.2 images per sample on average vs 2.6 for ICoT; removing DVTI or PVTG drops M3CoT performance by 14.4% and 13.8% respectively.

Surprising findings

• Confidence increased after image insertion in 80.7% of samples for DaP-ICoT vs 46.4% for ICoT.

Limitations

• Confidence threshold tau must be tuned on the M3CoT validation set, so generalization of the threshold is not established.
• Relies on SAM2 segmentation and attention-based similarity, so object-level pool quality depends on the segmenter.

Experimental setup

Datasets: M3CoT ScienceQA MME

Benchmarks: M3CoT ScienceQA MME

Metrics: accuracy MME score (perception + cognition) token consumption image insertion count

Baselines: Direct MMCoT DDCoT SCAFFOLD CCoT ICoT

Builds on: ICoT (attention-driven selection) Segment Anything Model 2 Multimodal-CoT DDCoT SCAFFOLD

Closest papers by meaning

• Visual Thoughts: A Unified Perspective of Understanding Multimodal Chain-of-Thought (NeurIPS 2025 · similarity 0.92)
• Interleaved-Modal Chain-of-Thought (CVPR 2025 · similarity 0.91)
• CoMT: A Novel Benchmark for Chain of Multi-modal Thought on Large Vision-Language Models (AAAI 2025 · similarity 0.90)
• Imagine While Reasoning in Space: Multimodal Visualization-of-Thought (ICML 2025 · similarity 0.90)
• CMMCoT: Enhancing Complex Multi-Image Comprehension via Multi-Modal Chain-of-Thought and Memory Augmentation (AAAI 2026 · similarity 0.90)
• ThinkMorph: Emergent Properties in Multimodal Interleaved Chain-of-Thought Reasoning (ICLR 2026 · similarity 0.89)