Research map / Vision-language and multimodal / Multimodal LLMs

VisionReasoner: Unified Reasoning-Integrated Visual Perception via Reinforcement Learning

ICLR 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Reinforcement learning for visual reasoning · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes VisionReasoner, a Qwen2.5-VL-7B plus SAM2 system trained with GRPO and a unified reward (format, multi-object IoU/L1 with Hungarian matching) to reason through detection, segmentation, and counting in one shared model.

Problem

Visual perception tasks (detection, segmentation, counting) are usually handled by task-specific models or separately RL-trained LVLMs, limiting scalability and generalization; multi-object outputs also complicate RL reward computation because predictions must be matched to ground truth.

Approach

A single LVLM generates a think-then-answer reasoning trace and outputs boxes and points; a unified reward (thinking/answer/non-repeat format rewards plus Hungarian-matched multi-object IoU and L1 rewards) is optimized with GRPO on about 7k samples drawn from LVIS, RefCOCOg, gRefCOCO, and LISA++, and SAM2 converts boxes to masks for segmentation.

Key results

• VisionReasoner-7B reaches 37.7 AP on COCO val versus 29.2 for Qwen2.5-VL-7B (reported relative gain of 29.1%).
• On ReasonSeg val, VisionReasoner-7B reaches 66.3 gIoU versus 56.9 for Qwen2.5-VL-7B, and 71.0 average segmentation, outperforming Seg-Zero-7B's 69.8.
• Human evaluation on ReasonSeg-val finds 97.0% image consistency and 90.5% answer consistency of reasoning traces.

Surprising findings

• Using a Hungarian-matched batch reward gives a 4x speedup in matching for 30 objects, while the paper reports no quantitative accuracy degradation.
• Qwen2.5 instruction-following falls below 50% accuracy on the paper's task-routing classification, while an RL-trained router performs much better.

Limitations

• Performance lags some task-specific detectors on COCO (e.g., DQ-DETR 50.2 AP).
• Relative gains are computed against the baseline Qwen2.5-VL and LVLM AP is underestimated by the bbox-area confidence proxy.
• Full evaluation of all listed task types is deferred; only 10 tasks are evaluated.

Experimental setup

Datasets: LVIS RefCOCOg gRefCOCO LISA++

Benchmarks: COCO RefCOCO RefCOCO+ RefCOCOg ReasonSeg PixMo-Count CountBench

Metrics: AP bbox AP@0.5 gIoU cIoU count accuracy

Baselines: Qwen2.5-VL-7B Qwen2-VL-7B InternVL2-8B Shikra-7B Seg-Zero-7B LISA-7B GLaMM-7B PixelLM-7B LLaVA-OneVision-7B DQ-DETR

Builds on: GRPO Visual-RFT Seg-Zero DeepSeek-R1-Zero DAPO

Closest papers by meaning

• RegionReasoner: Region-Grounded Multi-Round Visual Reasoning (ICLR 2026 · similarity 0.90)
• Artemis: Structured Visual Reasoning for Perception Policy Learning (ICML 2026 · similarity 0.90)
• From Seeing to Thinking: Decoupling Perception and Reasoning Improves Post-Training of Vision-Language Models (ICML 2026 · similarity 0.89)
• Unleashing Perception-Time Scaling to Multimodal Reasoning Models (ICLR 2026 · similarity 0.89)
• SPARC: Separating Perception And Reasoning Circuits for Test-time Scaling of VLMs (ICML 2026 · similarity 0.89)
• Bad Seeing or Bad Thinking? Rewarding Perception for Multimodal Reasoning (ICML 2026 · similarity 0.89)