Research map / Vision-language and multimodal / Multimodal LLMs

Do multimodal models imagine electric sheep?

NeurIPS 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Chain-of-thought multimodal reasoning · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Fine-tunes Qwen3.5 9B with open-loop behavior cloning on twelve spatial puzzles, shows intermediate visual states emerge in activations, and improves solve rate via visual chain-of-thought tokens (89.5% average).

Problem

Whether multimodal models form internal visual world models (mental imagery) when trained to solve multi-step spatial puzzles, and how to sharpen and use that imagery to improve spatial reasoning.

Approach

Open-loop behavior cloning of action sequences from the initial image induces decodable intermediate visual states in VLM activations; the paper then adds an auxiliary visual prediction loss and interleaved visual tokens (16 or 64 per step) in the chain of thought to strengthen and use these images.

Key results

• Stock Qwen3.5 9B reaches 13% average solve rate; action-only behavior cloning reaches 83%.
• Visual chain-of-thought with 16 visual tokens per step (64x64) reaches 89.5% average, versus 89.0% for 128x128 tokens.
• Textual CoT with JSON scene graphs reaches 78.7% average, about 10.8 points below the best visual CoT method.

Surprising findings

• Reconstruction from frozen action-only activations reveals intermediate world states without any visual supervision on the LLM.
• Anagram is an exception where base model has better but uninformative reconstructions (mean-image approximation).

Limitations

• Mental image reconstructions are imperfect with color errors and limited fidelity.
• BAGEL adaptation failed to converge, so the unified model comparison is weak.
• Evaluated on synthetic puzzles with expert solvers, limiting generality to natural visual reasoning.

Experimental setup

Datasets: procedurally generated puzzles from custom Rust game engines with expert solvers

Benchmarks: twelve-task spatial puzzle suite (3D/2D shape matching, mental rotation, jigsaw, tangram, sokoban, bloxorz, rush hour, a

Metrics: solve rate reconstruction fidelity of visual decoder

Baselines: Qwen3.5 9B base Action Supervision Textual CoT BAGEL-7B-MoT

Builds on: chain-of-thought VQ-VAE LlamaGen masked autoencoders BAGEL

Closest papers by meaning

• Unfolding Spatial Cognition: Evaluating Multimodal Models on Visual Simulations (ICLR 2026 · similarity 0.90)
• When Visualizing is the First Step to Reasoning: MIRA, a Benchmark for Visual Chain-of-Thought (CVPR 2026 · similarity 0.90)
• Machine Mental Imagery: Empower Multimodal Reasoning with Latent Visual Tokens (CVPR 2026 · similarity 0.89)
• Zebra-CoT: A Dataset for Interleaved Vision-Language Reasoning (ICLR 2026 · similarity 0.89)
• Reinforcing Spatial Reasoning in Vision-Language Models with Interwoven Thinking and Visual Drawing (NeurIPS 2025 · similarity 0.89)
• Imagine While Reasoning in Space: Multimodal Visualization-of-Thought (ICML 2025 · similarity 0.88)