AI Paper Atlas

Research map / Vision-language and multimodal / Multimodal LLMs

VisualPRM400K: An Effective Dataset for Training Multimodal Process Reward Models

Weiyun Wang, Zhangwei Gao, Lianjie Chen, Zhe Chen, Jinguo Zhu, Xiangyu Zhao, Yangzhou Liu, Yue Cao, Shenglong Ye, Xizhou Zhu, Lewei Lu, Haodong Duan and 3 more

ICLR 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Multimodal LLM benchmarks

Read the paperCheck your idea against 137,217 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Constructs VisualPRM400K, a ~400K multimodal process-supervision dataset, and trains VisualPRM, a multimodal process reward model that scores each reasoning step for Best-of-N selection.

Problem

Multimodal large language models need step-level verification for reasoning, but training multimodal process reward models requires large-scale process supervision data that is scarce.

Approach

Builds a large (~400K) multimodal process supervision dataset and uses it to train a process reward model that estimates the value score of each reasoning step, applied to improve Best-of-N reasoning across several MLLM families and scales.

Key results

Measured effects reported in the paper

What was changedWhereMeasuredEffect
apply VisualPRM as Best-of-8 criticInternVL2.5-8B · 7 reasoning benchmarks overalloverall accuracy: 32.8 -> 41.2Improved
apply VisualPRM as critic to the larger InternVL2.5-78BInternVL2.5-78B · 7 reasoning benchmarks overalloverall accuracy: 46.0 -> 51.9Improved
use VisualPRM rather than prompting InternVL2.5 models as critics (ablation)Best-of-8 · 7 reasoning benchmarks overalloverall accuracy: 33.2 (InternVL2.5-8B critic) -> 41.1Improved
use PRM instead of outcome reward model at Best-of-8 (ablation)InternVL2.5-8B · 7 reasoning benchmarks overalloverall accuracy: 39.7 -> 41.2Improved
increase N from 8 to 128 with ORM critic (ablation)VisualORM · 7 reasoning benchmarks overalloverall accuracy: 39.7 -> 39.7No clear effect
increase N from 8 to 128 with VisualPRM critic (ablation)VisualPRM · 7 reasoning benchmarks overalloverall accuracy: 41.2 -> 44.0Improved
value-based instead of advantage-based PRM (ablation)VisualPRM · 7 reasoning benchmarks overall, Best-of-8overall accuracy: 37.4 -> 41.1Improved
supervise all steps instead of stopping at first incorrect step (ablation)VisualPRM · 7 reasoning benchmarks overall, Best-of-8overall accuracy: 40.6 -> 41.1Improved
aggregate step scores by max instead of average (ablation)VisualPRM · 7 reasoning benchmarks overall, Best-of-8overall accuracy: 41.1 -> 35.9Hurt
raise step-correctness threshold from 0 to 0.25 (ablation)VisualPRM · 7 reasoning benchmarks overall, Best-of-8overall accuracy: 41.1 -> 40.0Hurt
raise sampling temperature from 0.7 to 1.3 (ablation)InternVL2.5-8B policy · 7 reasoning benchmarks overall, Best-of-8overall accuracy: 41.1 -> 38.6Hurt
allow more steps in data construction (1 to 12) (ablation)VisualPRM data · 7 reasoning benchmarks overall, Best-of-8overall accuracy: 39.7 -> 41.2Improved

Experimental setup

Datasets: VisualPRM400K

Metrics: Best-of-N reasoning performance

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy