Research map / Vision-language and multimodal / Multimodal LLMs
VisualPRM400K: An Effective Dataset for Training Multimodal Process Reward Models
Weiyun Wang, Zhangwei Gao, Lianjie Chen, Zhe Chen, Jinguo Zhu, Xiangyu Zhao, Yangzhou Liu, Yue Cao, Shenglong Ye, Xizhou Zhu, Lewei Lu, Haodong Duan and 3 more
Read the paperCheck your idea against 137,217 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Constructs VisualPRM400K, a ~400K multimodal process-supervision dataset, and trains VisualPRM, a multimodal process reward model that scores each reasoning step for Best-of-N selection.
Problem
Multimodal large language models need step-level verification for reasoning, but training multimodal process reward models requires large-scale process supervision data that is scarce.
Approach
Builds a large (~400K) multimodal process supervision dataset and uses it to train a process reward model that estimates the value score of each reasoning step, applied to improve Best-of-N reasoning across several MLLM families and scales.
Key results
- VisualPRM improves Best-of-N reasoning performance of three types of MLLMs across four model scales (per abstract; no specific numbers stated).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| apply VisualPRM as Best-of-8 critic | InternVL2.5-8B · 7 reasoning benchmarks overall | overall accuracy: 32.8 -> 41.2 | Improved |
| apply VisualPRM as critic to the larger InternVL2.5-78B | InternVL2.5-78B · 7 reasoning benchmarks overall | overall accuracy: 46.0 -> 51.9 | Improved |
| use VisualPRM rather than prompting InternVL2.5 models as critics (ablation) | Best-of-8 · 7 reasoning benchmarks overall | overall accuracy: 33.2 (InternVL2.5-8B critic) -> 41.1 | Improved |
| use PRM instead of outcome reward model at Best-of-8 (ablation) | InternVL2.5-8B · 7 reasoning benchmarks overall | overall accuracy: 39.7 -> 41.2 | Improved |
| increase N from 8 to 128 with ORM critic (ablation) | VisualORM · 7 reasoning benchmarks overall | overall accuracy: 39.7 -> 39.7 | No clear effect |
| increase N from 8 to 128 with VisualPRM critic (ablation) | VisualPRM · 7 reasoning benchmarks overall | overall accuracy: 41.2 -> 44.0 | Improved |
| value-based instead of advantage-based PRM (ablation) | VisualPRM · 7 reasoning benchmarks overall, Best-of-8 | overall accuracy: 37.4 -> 41.1 | Improved |
| supervise all steps instead of stopping at first incorrect step (ablation) | VisualPRM · 7 reasoning benchmarks overall, Best-of-8 | overall accuracy: 40.6 -> 41.1 | Improved |
| aggregate step scores by max instead of average (ablation) | VisualPRM · 7 reasoning benchmarks overall, Best-of-8 | overall accuracy: 41.1 -> 35.9 | Hurt |
| raise step-correctness threshold from 0 to 0.25 (ablation) | VisualPRM · 7 reasoning benchmarks overall, Best-of-8 | overall accuracy: 41.1 -> 40.0 | Hurt |
| raise sampling temperature from 0.7 to 1.3 (ablation) | InternVL2.5-8B policy · 7 reasoning benchmarks overall, Best-of-8 | overall accuracy: 41.1 -> 38.6 | Hurt |
| allow more steps in data construction (1 to 12) (ablation) | VisualPRM data · 7 reasoning benchmarks overall, Best-of-8 | overall accuracy: 39.7 -> 41.2 | Improved |
Experimental setup
Datasets: VisualPRM400K
Metrics: Best-of-N reasoning performance
Closest papers by meaning
- Unlocking Multimodal Mathematical Reasoning via Process Reward Model (NeurIPS 2025 · similarity 0.92)
- DreamPRM: Domain-reweighted Process Reward Model for Multimodal Reasoning (NeurIPS 2025 · similarity 0.91)
- Training Data Efficiency in Multimodal Process Reward Models (ICML 2026 · similarity 0.91)
- What, Whether and How? Unveiling Process Reward Models for Thinking with Images Reasoning (AAAI 2026 · similarity 0.89)
- Rewarding Progress: Scaling Automated Process Verifiers for LLM Reasoning (ICLR 2025 · similarity 0.88)
- Perception-R1: Advancing Multimodal Reasoning Capabilities of MLLMs via Visual Perception Reward (ICLR 2026 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
