Research map / Vision-language and multimodal / Multimodal LLMs

ThinkMorph: Emergent Properties in Multimodal Interleaved Chain-of-Thought Reasoning

ICLR 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Chain-of-thought multimodal reasoning · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Fine-tunes the unified model Bagel-7B on ~24K interleaved text-image reasoning traces (ThinkMorph) and shows gains on vision-centric benchmarks plus emergent manipulation, mode-switching and test-time scaling behaviors.

Problem

Current unified multimodal models lack a generalizable recipe for interleaved text-image chain-of-thought that mutually advances visual and verbal reasoning, especially on vision-centric tasks.

Approach

Treat text and image thoughts as complementary rather than isomorphic modalities; build task-specific interleaved traces (text, then image manipulation, then text verification) for four tasks and fine-tune a unified model on them with joint MSE image loss and NLL text loss.

Key results

• ThinkMorph averages a 34.74% relative improvement over Bagel-7B on the three vision-centric tasks, reaching 86.67% on Spatial Navigation (VSP) vs 0.83% base.
• ThinkMorph reaches 52.67% on SAT versus 49.33% for InternVL3.5-38B and 28.00% for GPT-4o, and 80.33% on MMVP, matching Gemini 2.5 Flash.
• Interleaved reasoning outperforms text-only and visual-only modes by 5.33% on average; the model autonomously switches to text-only in 5.3% of MMVP cases with higher accuracy (81.25%).

Surprising findings

• Despite training only on interleaved traces, the model autonomously switches to text-only reasoning on some out-of-domain inputs, and those switched samples are more accurate.
• On in-domain ChartQA, text-only reasoning slightly outperforms interleaved reasoning, indicating visual highlighting can be unnecessary for some tasks.

Limitations

• Interleaved reasoning is about 3x more computationally expensive than text-only due to image generation.
• Emergent behaviors are described qualitatively and on limited benchmarks; some unseen manipulations account for only small fractions of operations.
• Relies on proprietary GPT-4.1/GPT-5/Gemini models for data generation and judging.

Experimental setup

Datasets: SAT ADE20K Omni3D Visual CoT GQA VSR Open Images ChartQA Refocus OpenAI Gym Frozen Lake

Benchmarks: VSP-main VisPuzzle ChartQA VStar BLINK-J MMVP SAT BLINK CV-Bench

Metrics: accuracy

Baselines: Bagel-7B Janus-Pro-7B Chameleon-7B InternVL3.5-8B InternVL3.5-38B Qwen2.5-VL-7B Qwen2.5-VL-72B GPT-4o GPT-5 Gemini 2.5 Flash

Builds on: Bagel Visual CoT MVoT

Closest papers by meaning

• AStar: Boosting Multimodal Reasoning with Automated Structured Thinking (AAAI 2026 · similarity 0.94)
• Visual Thoughts: A Unified Perspective of Understanding Multimodal Chain-of-Thought (NeurIPS 2025 · similarity 0.93)
• DiffThinker: Towards Generative Multimodal Reasoning with Diffusion Models (ICML 2026 · similarity 0.92)
• ROVER: Benchmarking Reciprocal Cross-Modal Reasoning for Omnimodal Generation (ICLR 2026 · similarity 0.92)
• UniVLR: Unifying Text and Vision in Visual Latent Reasoning for Multimodal LLMs (NeurIPS 2026 · similarity 0.92)
• TVI-CoT: Text-Visual Interleaved Chain-of-Thought Reasoning for Multimodal Understanding (ICML 2026 · similarity 0.92)