Research map / Large language models / Reasoning and chain of thought

ContextPRM: Leveraging Contextual Coherence for multi-domain Test-Time Scaling

ICLR 2026 · Large language models › Reasoning and chain of thought · approach: Process reward models · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes ContextPRM, a process reward model trained with context-coherence labels and context-paired inputs (previous step as context) on Llama-PRM800K via LoRA, evaluated for multi-domain test-time scaling on MMLU-Pro.

Problem

Process reward models trained mainly on math transfer poorly to non-mathematical reasoning domains, where they lack domain-specific training data and knowledge-based patterns.

Approach

Shift PRM learning from verifying isolated step correctness to modeling contextual coherence between consecutive steps: each step is paired with its preceding step as context, labeled with a stricter coherence-based annotation standard, and trained with a per-step cross-entropy loss.

Key results

• ContextPRM achieves a 6.5% average accuracy improvement over majority voting in non-math-adjacent domains under weighted majority voting, versus 2.2% for VersaPRM.
• Under BoN sampling, ContextPRM gains 6.3% over majority voting in non-math-adjacent domains.
• Ablation: context-based training alone gives +1.07% and context-based labeling alone gives +0.84% in non-math-adjacent domains; full ContextPRM gives +4.3%.

Surprising findings

• ContextPRM shows a 2.2% accuracy decrease in the math-adjacent domain relative to VersaPRM, described as a trade-off for generalization.
• Re-annotating with the coherence standard modifies the first-error location for 42.82% of CoTs, including 24.67% of CoTs originally labeled fully correct.

Limitations

• Trade-off with reduced math-adjacent accuracy relative to VersaPRM.
• Annotation relies on gpt-4o-mini judgments for the first-incorrect-step comparison; the annotation pipeline's reliability is not independently validated in the text.
• Evaluated on a single benchmark (MMLU-Pro-CoT-Eval) with one generator's CoTs.

Experimental setup

Datasets: VersaPRM training data PRM800K MMLU-Pro-CoT-Eval (Unlabeled)

Benchmarks: MMLU-Pro-CoT-Eval (Unlabeled)

Metrics: accuracy accuracy improvement over majority voting

Baselines: VersaPRM Qwen2.5-Math-PRM Math-Shepherd RLHFlow-Deepseek Llama-PRM800K Majority Voting

Builds on: VersaPRM Shepherd Llama-PRM800K

Closest papers by meaning

• From Mathematical Reasoning to Code: Generalization of Process Reward Models in Test-Time Scaling (AAAI 2026 · similarity 0.91)
• Rethinking Reward Models for Multi-Domain Test-Time Scaling (NeurIPS 2026 · similarity 0.91)
• VersaPRM: Multi-Domain Process Reward Model via Synthetic Reasoning Data (ICML 2025 · similarity 0.90)
• Unlocking Multimodal Mathematical Reasoning via Process Reward Model (NeurIPS 2025 · similarity 0.89)
• GenPRM: Scaling Test-Time Compute of Process Reward Models via Generative Reasoning (AAAI 2026 · similarity 0.89)
• Know What You Don't Know: Uncertainty Calibration of Process Reward Models (NeurIPS 2025 · similarity 0.88)