Research map / Vision-language and multimodal / Multimodal LLMs
Discriminative Visual Process Rewards for Scaling Thinking at Test-Time with Images
Bo-Wen Yin, Qize Yang, Boyuan Sun, Xihan Wei, Qibin Hou
Read the paperCheck your idea against 137,217 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a discriminative process reward model that scores intermediate visual steps (cropping, annotation, localization, sketches) in thinking-with-images reasoning, enabling test-time scaling for multimodal LLMs.
Problem
Existing multimodal process reward models evaluate only textual reasoning and cannot judge the correctness of intermediate visual steps in "thinking with images" reasoning, leaving a gap for verifying and scaling visual reasoning at test time.
Approach
A discriminative visual process reward model that judges the correctness of intermediate visual actions (e.g., cropping, annotation, localization, sketching) rather than only textual reasoning steps, used to guide test-time scaling of thinking with images.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use DiscPRM instead of VisualPRM for Best-of-8 selection | Deepeyes · Perception benchmarks | average accuracy: 71.5 -> 75.5 | Improved |
| use DiscPRM instead of VisualPRM for Best-of-8 selection | Thyme-7B · Reasoning benchmarks | average accuracy: 43.8 -> 47.4 | Improved |
| use DiscPRM instead of VisualPRM for Best-of-8 selection | Thyme-7B · General benchmarks | average accuracy: 72.6 -> 75.9 | Improved |
| use DiscPRM rather than Self-Consistency majority voting | Deepeyes · MathVision | accuracy: 27.3 -> 36.4 | Improved |
| use Self-Consistency majority voting over 8 samples | Thyme-7B · V* | accuracy: 82.2 -> 82.8 | No clear effect |
| use value-based instead of advantage-based formulation (ablation) | DiscPRM · MathVista | accuracy: 72.4 -> 75.5 | Improved |
| supervise all steps instead of stopping at first incorrect step (ablation) | DiscPRM · MathVista | accuracy: 73.5 -> 75.5 | Improved |
| supervise all steps instead of stopping at first incorrect step (ablation) | DiscPRM · VABench | macro F1: 63.4 -> 65.9 | Improved |
| average step scores instead of max pooling (ablation) | DiscPRM · MathVista | accuracy: 73.1 -> 75.5 | Improved |
| average step scores instead of min pooling (ablation) | DiscPRM · MathVista | accuracy: 73.5 -> 75.5 | Improved |
| raise data sampling temperature from 0.7 to 1.3 (ablation) | DiscPRM · V* | accuracy: 89.5 -> 88.3 | Hurt |
| lower data sampling temperature from 0.7 to 0.4 (ablation) | DiscPRM · V* | accuracy: 89.5 -> 88.8 | No clear effect |
Experimental setup
Builds on: process reward models thinking with images
Closest papers by meaning
- What, Whether and How? Unveiling Process Reward Models for Thinking with Images Reasoning (AAAI 2026 · similarity 0.93)
- When Visualizing is the First Step to Reasoning: MIRA, a Benchmark for Visual Chain-of-Thought (CVPR 2026 · similarity 0.90)
- RewardMap: Tackling Sparse Rewards in Fine-grained Visual Reasoning via Multi-Stage Reinforcement Learning (ICLR 2026 · similarity 0.89)
- Thinking with Programming Vision: Towards a Unified View for Thinking with Images (CVPR 2026 · similarity 0.89)
- VLM-R³: Region Recognition, Reasoning, and Refinement for Enhanced Multimodal Chain-of-Thought (NeurIPS 2025 · similarity 0.89)
- Look on Demand: A Cognitive Scheduling Framework for Visual Evidence Acquisition in Multimodal Reasoning (ICML 2026 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
