Research map / Vision-language and multimodal / Multimodal LLMs

Improving Vision-language Models with Perception-centric Process Reward Models

CVPR 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Reinforcement learning for visual reasoning · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes Perceval, a perception-centric process reward model that flags hallucinated spans in VLM responses, used both to assign token-level penalties in GRPO training and to drive truncate-and-regenerate test-time scaling.

Problem

Outcome-level rewards in RLVR are too coarse to locate perception errors within VLM reasoning chains, causing credit-assignment and sparse-reward problems for visual reasoning.

Approach

A PRM (Perceval) extracts image-related claims from a response, checks them against visual evidence, and returns erroneous spans; these spans get token-level advantage penalties in GRPO and are truncated and regenerated at inference.

Key results

• On Qwen2.5-VL-3B, the method lifts V* overall from 80.10 (GRPO) to 83.25 and MathVision from 23.36 to 26.32.
• On Qwen2.5-VL-7B, the method reaches 86.39 on V* overall vs 84.29 for GRPO, and 54.49 on BLINK vs 53.55.
• Truncate test-time scaling with k=16 reaches 89.53 on V* overall vs 85.86 for major voting on the 3B policy.

Surprising findings

• Perception-focused RL with the PRM on visual-search data transfers gains to math and chart benchmarks (MathVision, ChartQA) even though those domains were not PRM-supervised.
• Overly large penalty strength (alpha=0.3) degrades performance, attributed to penalizing benign tokens within flagged spans.

Limitations

• Perceval flags spans by exact string match, and penalizing whole flagged substrings can penalize benign tokens at high alpha.
• Feedback/reflection strategy underperforms because training data lacks reflective examples.
• Reward hacking assessed only through a hallucination-proportion curve, not a stronger adversarial test.

Experimental setup

Datasets: visual search datasets DeepEyes SFT data SophiaVL-R1 data

Benchmarks: V* BLINK MMStar MME-RealWorld RealWorldQA MathVision MathVista ChartQA

Metrics: accuracy relaxed accuracy

Baselines: VLM-R1 LMM-R1 R1-VL Perception-R1 Jigsaw-R1 DeepEyes PixelReasoner Vision-R1 VL-Rethinker VLAA-Thinker OpenVLThinker MM-Eureka

Builds on: GRPO RLVR DeepEyes SophiaVL-R1

Closest papers by meaning

• PDCR: Perception-Decomposed Confidence Reward for Vision-Language Reasoning (CVPR 2026 · similarity 0.92)
• Bad Seeing or Bad Thinking? Rewarding Perception for Multimodal Reasoning (ICML 2026 · similarity 0.91)
• PeRL: Permutation-Enhanced Reinforcement Learning for Interleaved Vision-Language Reasoning (NeurIPS 2025 · similarity 0.90)
• Perception-R1: Advancing Multimodal Reasoning Capabilities of MLLMs via Visual Perception Reward (ICLR 2026 · similarity 0.90)
• Perceptual-Evidence Anchored Reinforced Learning for Multimodal Reasoning (CVPR 2026 · similarity 0.90)
• Unveiling Chain of Step Reasoning for Vision-Language Models with Fine-grained Rewards (NeurIPS 2025 · similarity 0.90)