Research map / Vision-language and multimodal / Multimodal LLMs

Monet: Reasoning in Latent Visual Space Beyond Image and Language

CVPR 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Chain-of-thought multimodal reasoning

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes Monet, a three-stage distillation-based SFT pipeline plus the VLPO policy-gradient method so an MLLM can reason with continuous latent visual embeddings, trained on a curated 125K interleaved CoT dataset.

Problem

Existing thinking-with-images methods rely on external tools or cropping, and prior latent visual reasoning approaches suffer from costly latent-vision alignment, weak supervision of latent embeddings, and GRPO not optimizing latent tokens.

Approach

Generate continuous latent embeddings as intermediate visual thoughts, supervised by aligning observation-token hidden states with a teacher that sees the auxiliary images, with a controlled attention flow and latent-only gradients; then VLPO treats latent embeddings as Gaussian-policy actions so outcome rewards directly update them.

Key results

• Monet-7B reaches 83.25 on V*, 71.00 on HRBench4K, 68.00 on HRBench8K, improving Qwen2.5-VL-7B by 4.25-9.75 points across real-world benchmarks.
• On the out-of-distribution VisualPuzzles benchmark, Monet-7B scores 35.02 overall versus 32.71 for Qwen2.5-VL-7B and 30.99 for vanilla SFT + GRPO.
• Ablation shows Monet-SFT + GRPO does not reliably improve over Monet-SFT, while adding VLPO raises V* from 82.20 to 83.25 and VisualPuzzles from 30.48 to 35.02.

Surprising findings

• GRPO on top of SFT often underperforms the SFT model when latent embeddings are used at test time, suggesting it mainly strengthens text reasoning.
• Removing latent-only backpropagation from the alignment loss causes a sharp drop (V* from 82.20 to 46.07).

Limitations

• Multi-stage SFT pipeline increases training complexity and overhead.
• Reward design alternatives for latent visual reasoning were not explored.
• Monet-SFT-125K relies on API models (DeepSeek-V3.1, Gemini 2.5 Pro) for observation labeling.

Experimental setup

Datasets: Monet-SFT-125K ReFocus CogCoM Zebra-CoT Visual-CoT Thyme-RL

Benchmarks: V* HRBench4K HRBench8K MME-RealWorld-Lite VisualPuzzles

Metrics: accuracy

Baselines: Qwen2.5-VL-7B Vanilla SFT Vanilla SFT + GRPO Deepeyes LVR GPT-4o Gemini 2.5 Pro Pangea-7B LLaVA-OneVision-72B

Builds on: GRPO LVR Deepeyes Qwen2.5-VL

Closest papers by meaning

• Latent Visual Reasoning (ICLR 2026 · similarity 0.93)
• SLVR: Structured Latent Visual Reasoning via Human-like Reasoning Flows (NeurIPS 2026 · similarity 0.93)
• Leveraging Latent Visual Reasoning in Silence (NeurIPS 2026 · similarity 0.93)
• Fill the GAP: A Granular Alignment Paradigm for Visual Reasoning in Multimodal Large Language Models (NeurIPS 2026 · similarity 0.92)
• ATLAS: Agentic or Latent Visual Reasoning? One Word is Enough for Both (NeurIPS 2026 · similarity 0.92)
• Semantic-Enriched Latent Visual Reasoning (ICML 2026 · similarity 0.92)