Research map / Vision-language and multimodal / Multimodal LLMs
Mental Imagery That Matters: Curing Latent Collapse in Multimodal Reasoning
Romy Luo, Rohan Dugad, Zihui (Sherry) Xue, Alex Dimakis, Kristen Grauman
Read the paperCheck your idea against 137,217 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Identifies latent collapse in latent multimodal reasoning, where interleaved continuous visual latent blocks lose their visual content, and proposes a fix to keep internal visual workspaces meaningful.
Problem
Latent multimodal reasoning, which interleaves text with continuous visual latent blocks to simulate mental imagery, fails in practice because the visual latents collapse and do not carry meaningful visual state.
Approach
Diagnoses the collapse of continuous visual latents in interleaved multimodal reasoning and addresses it so that latent blocks act as a real internal visual workspace without decoding to pixels. Details beyond the abstract are not available.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add AnchorSim Stage 2 (contrastive anchoring + EMA distillation) on top of the interleaved Stage 1 model | Qwen-2.5-VL-7B (interleaved) · Average of 9 benchmarks | average accuracy: 59.7 -> 62.4 | Improved |
| train AnchorSim vs plain base model | Qwen2.5-VL-7B · Average of 9 benchmarks | average accuracy: 48.9 -> 62.4 | Improved |
| train AnchorSim vs interleaved Stage 1 | Qwen-2.5-VL-7B (interleaved) · VSP | accuracy: 74.8 -> 78.0 | Improved |
| train AnchorSim vs interleaved Stage 1 | Qwen-2.5-VL-7B (interleaved) · V* | accuracy: 79.1 -> 82.7 | Improved |
| add AnchorSim objective | Qwen-2.5-VL-7B (interleaved) · CV-Bench 3D | accuracy: 77.3 -> 86.4 | Improved |
| add anchor loss on top of distillation-only training (ablation) | AnchorSim · V* | accuracy: +4.2 points | Improved |
| add L_distill loss to base model (ablation) | Qwen2.5-VL-7B · BLINK, V*, CV-Bench | accuracy | Improved |
| increase latent block size from 8 to 16 tokens (ablation) | AnchorSim · BLINK and CV-Bench | accuracy | Hurt |
| increase latent block size from 8 to 16 tokens (ablation) | AnchorSim · V* | accuracy: small improvement | Improved |
| add latent tokens vs no-latent Qwen base (ablation) | Qwen2.5-VL-7B · BLINK, V*, CV-Bench | accuracy | Improved |
| perturb visual latents (Drop/Cross/Shuffle) in existing latent-reasoning MLLMs (ablation) | Mirage, SkiLa, CoVT, Monet · pivot study, 100 samples per model | final-answer log-probability: Mirage Drop slope vision 0.87 vs text 0.59 | No clear effect |
| apply AnchorSim to make latents consequential in closed-loop perturbation (ablation) | Qwen-interleaved vs AnchorSim · closed-loop flip rate | flip rate of final answer: 15-17% -> 29-37% | Improved |
Closest papers by meaning
- Machine Mental Imagery: Empower Multimodal Reasoning with Latent Visual Tokens (CVPR 2026 · similarity 0.93)
- Imagination Helps Visual Reasoning, But Not Yet in Latent Space (ICML 2026 · similarity 0.92)
- Leveraging Latent Visual Reasoning in Silence (NeurIPS 2026 · similarity 0.92)
- ThinkMorph: Emergent Properties in Multimodal Interleaved Chain-of-Thought Reasoning (ICLR 2026 · similarity 0.91)
- Fill the GAP: A Granular Alignment Paradigm for Visual Reasoning in Multimodal Large Language Models (NeurIPS 2026 · similarity 0.91)
- MentisOculi: Revealing the Limits of Reasoning with Mental Imagery (ICML 2026 · similarity 0.91)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
