Research map / Vision-language and multimodal / Multimodal LLMs

Pixel Reasoner: Incentivizing Pixel Space Reasoning via Curiosity-Driven Reinforcement Learning

NeurIPS 2025 · Vision-language and multimodal › Multimodal LLMs · approach: Reinforcement learning for visual reasoning · 6 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces pixel-space reasoning in VLMs with zoom-in and select-frame operations, trained via template-based SFT on synthesized trajectories followed by GRPO with a curiosity-driven reward that encourages exploring visual operations.

Problem

Vision-language models reason only in text, which limits fine-grained visual analysis on information-rich images and videos; newly introduced visual operations are rarely adopted by VLMs, creating a "learning trap" during RL.

Approach

Equip VLMs with visual operations (CropImage zoom-in, SelectFrames) inside the reasoning chain; warm-start with SFT on template-synthesized trajectories including error-induced self-correction, then apply RL with a curiosity bonus that rewards pixel-space reasoning when its rate is below a threshold, plus an efficiency penalty on the number of visual operations.

Key results

• PixelReasoner-7B reaches 84.3% on V* Bench, exceeding Gemini-2.5-Pro (79.2%) and reported as best open-source result.
• Achieves 73.8% on TallyQA-complex and 67.8% on MVBench, and 84.0 ANLS on InfographicsVQA.
• Warm-start model without RL drops to 79.0 on V* and 59.0 on MVBench, showing RL is necessary for gains.

Measured effects reported in the paper

Surprising findings

• Zero-shot Qwen2.5-VL-7B with visual operations prompts has lower accuracy when using pixel-space reasoning (23.2%) than textual reasoning (49.5%), and RaPR declines to zero under RL without incentives.
• Without curiosity bonus or with a single-pass-only warm start, the policy reward-hacks by invoking operations but ignoring errors.

Limitations

• Only two visual operations (zoom-in and select-frame) are supported.
• Performance bottlenecked by limited training data across tasks and content.

Experimental setup

Datasets: SA1B FineWeb STARQA InfographicVQA

Benchmarks: V* Bench TallyQA-complex MVBench InfographicsVQA

Metrics: accuracy ANLS

Baselines: GPT-4o Gemini-2.5-Pro Gemini-2.0-Flash Qwen2.5-VL-7B Video-R1 Visual Sketchpad SEAL Gemma3-27B

Builds on: GRPO Qwen2.5-VL VL-Rethinker selective sample replay curiosity-driven exploration

Closest papers by meaning

• Reason-RFT: Reinforcement Fine-Tuning for Visual Reasoning of Vision Language Models (NeurIPS 2025 · similarity 0.91)
• RewardMap: Tackling Sparse Rewards in Fine-grained Visual Reasoning via Multi-Stage Reinforcement Learning (ICLR 2026 · similarity 0.91)
• Multi-step Visual Reasoning with Visual Tokens Scaling and Verification (NeurIPS 2025 · similarity 0.90)
• MiCo: Multi-image Contrast for Reinforcement Visual Reasoning (NeurIPS 2025 · similarity 0.90)
• More Thought, Less Accuracy? On the Dual Nature of Reasoning in Vision-Language Models (ICLR 2026 · similarity 0.90)
• PeRL: Permutation-Enhanced Reinforcement Learning for Interleaved Vision-Language Reasoning (NeurIPS 2025 · similarity 0.90)