Research map / Large language models / Reasoning and chain of thought

DreamPRM: Domain-reweighted Process Reward Model for Multimodal Reasoning

NeurIPS 2025 · Large language models › Reasoning and chain of thought · approach: Process reward models · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes DreamPRM, a bi-level domain-reweighted training framework for multimodal process reward models that learns per-dataset weights using a MMMU meta set, improving test-time scaling of MLLMs on math and general multimodal benchmarks.

Problem

Training multimodal process reward models (PRMs) suffers from distribution shift and dataset quality imbalance across multimodal reasoning datasets, which degrades PRM generalization and test-time scaling performance.

Approach

Learns trainable domain weights via bi-level optimization: the lower level fits the PRM on Monte Carlo step labels from weighted training domains, and the upper level updates the weights by minimizing an aggregation-function loss (sigmoid of summed log-odds of step scores) against final-answer correctness on a meta dataset.

Key results

• DreamPRM with InternVL-2.5-8B-MPO reaches 57.4 WeMath, 68.9 MathVista, 22.1 MathVision, 61.4 MMVet, 62.3 MMStar, beating Vanilla PRM by about 2-3 points on each benchmark.
• DreamPRM with o4-mini reports 85.2% on the MathVista leaderboard (top-1 as of Oct 15, 2025), up from 80.6% pass@1 and 84.2% for Vanilla PRM.
• Removing bi-level optimization lowers MathVista from 68.9 to 65.4 (-3.5) and removing the aggregation function loss lowers WeMath from 57.4 to 56.3.

Measured effects reported in the paper

Surprising findings

• Learned domain weights favor M3CoT and FigureQA (~1.5) while AI2D and IconQA fall below 0.8, so easy datasets are down-weighted.
• Removing structural thinking prompts improves MMStar slightly (61.6 vs 62.3 drop of only 0.7) but hurts MMVet by 3.9 points.

Limitations

• Assumes a fixed set of domains and does not reweight at instance level.
• Monte Carlo sampling for step labels is computationally heavy.
• No error bars reported due to resource limits.

Experimental setup

Datasets: AI2D ScienceQA M3CoT ChartQA DVQA MapQA FigureQA Geo170k Geometry3K UniGeo GeomVerse GeoS

Benchmarks: WeMath MathVista MathVision MMVet MMStar

Metrics: accuracy

Baselines: Self-consistency Self-correction ORM Vanilla PRM s1-PRM CaR-PRM Gemini-1.5-Pro GPT-4V LLaVA-OneVision-7B Qwen2-VL-7B

Builds on: Math-Shepherd DoReMi DOGE Betty DAPO LLaVA-CoT

Closest papers by meaning

• Unlocking Multimodal Mathematical Reasoning via Process Reward Model (NeurIPS 2025 · similarity 0.90)
• VersaPRM: Multi-Domain Process Reward Model via Synthetic Reasoning Data (ICML 2025 · similarity 0.88)
• Rethinking Reward Models for Multi-Domain Test-Time Scaling (NeurIPS 2026 · similarity 0.88)
• RBench: Graduate-level Multi-disciplinary Benchmarks for LLM & MLLM Complex Reasoning Evaluation (ICML 2025 · similarity 0.88)
• Vision-R1: Incentivizing Reasoning Capability in Multimodal Large Language Models (ICLR 2026 · similarity 0.87)
• Reasoning-Driven Multimodal LLM for Domain Generalization (ICLR 2026 · similarity 0.87)