Research map / Vision-language and multimodal / Multimodal LLMs

Synergizing Understanding and Generation with Interleaved Analyzing-Drafting Thinking

ICLR 2026 · Vision-language and multimodal › Multimodal LLMs · approach: VLM capability analysis · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes AD-Loop, an interleaved analyzing-drafting thinking paradigm where a unified VLM alternates textual thoughts with latent visual thoughts, trained by SFT then GRPO-based adaptive RL, improving understanding and generation benchmarks.

Problem

Unified vision-language models treat understanding and generation as parallel skills with little explicit interaction during task solving, so they fail to achieve synergy between comprehension and generation.

Approach

Interleave textual thoughts (analysis) with compact latent visual thoughts (drafting) inside a think trace, using density-peaks clustering of generation-encoder latents as visual-thought targets, then use RL with an adaptive bonus and inter/intra-group advantages to learn when to use AD-Loop versus text-only thinking.

Key results

• AD-Loop on BAGEL-7B reaches 57.3 on MMMU and 69.7 on MM-Vet, and 86% overall on GenEval, above BAGEL (82) and MindOmni (83).
• Adaptive interleaved thinking reaches 75.8 on MathVista and 0.89 SAT versus 61.5 and 0.63 for isolated thinking.
• On spatial visual planning level 6, implicit latent visual thoughts reach 0.47 versus 0.39 for Mirage and 0.03 for MVoT.

Surprising findings

• Explicit visual thoughts (T+I) perform only marginally differently from implicit latent thoughts in the ablation, and the authors attribute this partly to pixel-level noise in explicit supervision.
• Visual thoughts derived from the generation encoder outperform those from the understanding encoder on both understanding and generation benchmarks.

Limitations

• Over-drafts on simple attribute questions, adding unnecessary steps that can cause errors.
• Struggles with fine-grained generation details such as scene text, fingers, and numerically precise elements, possibly due to compact latent thoughts.
• Still struggles on problems requiring strict symbolic reasoning where AD-Loop may deviate from the optimal logical path.

Experimental setup

Datasets: CMoT Visual-CoT CoF VSP GoT-T2I X-to-Image

Benchmarks: POPE MME MMBench SEED GQA MMMU MM-Vet GenEval MathVista LogicVista WISE

Metrics: accuracy GenEval overall score WiScore

Baselines: LLaVA-v1.5 Janus-Pro BAGEL MMaDA MindOmni Emu3 Show-o

Builds on: GRPO BAGEL Janus-Pro Chain-of-Thought Mirage

Closest papers by meaning

• Thinking-while-Generating: Interleaving Textual Reasoning throughout Visual Generation (CVPR 2026 · similarity 0.89)
• ThinkMorph: Emergent Properties in Multimodal Interleaved Chain-of-Thought Reasoning (ICLR 2026 · similarity 0.88)
• When Visualizing is the First Step to Reasoning: MIRA, a Benchmark for Visual Chain-of-Thought (CVPR 2026 · similarity 0.87)
• Toward Cognitive Supersensing in Multimodal Large Language Models (NeurIPS 2026 · similarity 0.87)
• Janus-Pro-R1: Advancing Collaborative Visual Comprehension and Generation via Reinforcement Learning (NeurIPS 2025 · similarity 0.87)
• Visual Thoughts: A Unified Perspective of Understanding Multimodal Chain-of-Thought (NeurIPS 2025 · similarity 0.87)