Research map / Vision-language and multimodal / Multimodal LLMs
Insight-V: Exploring Long-Chain Visual Reasoning with Multimodal Large Language Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Insight-V generates long-chain visual reasoning data with a progressive generation and multi-granularity scoring pipeline, then trains a reasoning agent and a summary agent (multi-agent system) with SFT and iterative DPO to boost MLLM visual reasoning.
Problem
Long-chain visual reasoning in multimodal LLMs is limited by the lack of high-quality, long structured reasoning data and by training pipelines that do not effectively teach MLLMs to reason while retaining visual perception.
Approach
Decompose MLLM problem solving into a reasoning agent that produces long step-by-step rationales and a summary agent that judges those rationales and answers, with the reasoning agent refined by iterative DPO on model-generated, score-ranked reasoning paths.
Key results
- Insight-V-LLaVA (8B) reaches average 47.2 across 7 visual reasoning benchmarks, +7.0 over LLaVA-NeXT-LLaMA3 (40.2).
- Insight-V on the 7B in-house base model reaches average 51.6 (+2.9 over 48.7), with MMStar 61.5 and MathVista 59.9.
- On the ablation with the base model, iterative DPO raises average from 62.1 to 63.3 over the multi-agent system on four benchmarks, versus 62.7 for single DPO.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add the multi-agent reasoning and summarization system | LLaVA-NeXT-LLaMA3 · Visual reasoning benchmarks | Average: +7.0 | Helped |
| add the multi-agent reasoning and summarization system | Our Base Model · Visual reasoning benchmarks | Average: +2.9 | Helped |
| add iterative DPO after multi-agent training | LLaVA-NeXT-LLaMA3 · Visual reasoning benchmarks | Average: +2.6% | Helped |
| add iterative DPO after multi-agent training | Our Base Model · Visual reasoning benchmarks | Average: +1.0% | Helped |
| remove the reasoning agent (ablation) | Insight-V multi-agent system · Visual reasoning ablation benchmarks | Average: 62.1 -> 59.8 | Hurt |
| replace the multi-agent decomposition with direct SFT (ablation) | Our Base Model · Visual reasoning ablation benchmarks | Average: 62.1 -> 60.6 | Hurt |
| replace the multi-agent setup with multi-turn supervised training (ablation) | Our Base Model · Visual reasoning ablation benchmarks | Average: 62.1 -> 61.0 | Hurt |
| use RLAIF preference data instead of curated preference data (ablation) | Insight-V reasoning agent · Visual reasoning ablation benchmarks | Average: 62.1 -> 62.3 (RLAIF); 62.1 -> 62.7 (DPO) | Hurt |
| replace one-round DPO with iterative DPO (ablation) | Insight-V reasoning agent · Visual reasoning ablation benchmarks | Average: 62.7 -> 63.3 | Helped |
| increase reasoning-agent training data from 50K to 200K samples (ablation) | Insight-V reasoning agent · Six benchmarks | performance | Helped |
Surprising findings
- Vanilla direct SFT on chain-of-thought data yields only modest gains, and multi-turn single-model training is also suboptimal compared to the two-agent design.
- Reasoning agents trained on too little data (50K) perform worse than the baseline model.
Limitations
- Data generation depends on other large models for multi-granularity assessment, which is costly and inefficient.
- Training two same-size models may not scale; a smaller summary agent is suggested as future work.
- Still far from o1-like reasoning performance; limited reflection or inference-scaling mechanisms.
Experimental setup
Datasets: LLaVA-NeXT data Cauldron Cambrian-1 LLaVA-1.5 558K captioning RLAIF-V
Benchmarks: MMMU MMMU-Pro MMBench MME ChartQA MMStar MathVista TextVQA DocVQA OCRBench AI2D
Metrics: accuracy average score across benchmarks MME perception/cognition scores
Baselines: LLaVA-NeXT-LLaMA3 DeepSeek-VL VILA-1.5 Cambrian-1 InternLM-XComposer2 POINTS IXC-2.5 MiniCPM-V-2.6 Qwen2-VL Idefics3-LLaMA3 Ovis1.5-LLaMA3 RLAIF-V
Builds on: LLaVA-NeXT Chain-of-Thought prompting DPO iterative DPO RLAIF-V
Closest papers by meaning
- VisuLogic: A Benchmark for Evaluating Visual Reasoning in Multi-modal Large Language Models (ICLR 2026 · similarity 0.93)
- Vision-aligned Latent Reasoning for Multi-modal Large Language Model (ICML 2026 · similarity 0.93)
- Latent Visual Reasoning (ICLR 2026 · similarity 0.93)
- SLVR: Structured Latent Visual Reasoning via Human-like Reasoning Flows (NeurIPS 2026 · similarity 0.92)
- MMFineReason: Closing the Multimodal Reasoning Gap via Open Data-Centric Methods (NeurIPS 2026 · similarity 0.92)
- VisuRiddles: Fine-grained Perception is a Primary Bottleneck for Multimodal Large Language Models in Abstract Visual Reasoning (ICLR 2026 · similarity 0.92)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.