Research map / Large language models / Retrieval and knowledge
Process Reward Agents for Steering Knowledge-Intensive Reasoning
Jiwoong Sohn, Tomasz Sternal, Kenneth Styppa, Torsten Hoefler, Michael Moor
This paper tried a common change: Chain-of-thought prompting
add Self-Consistency to CoT on hard benchmarks · GPQA · accuracy: 60.8 -> 51.8 Hurt
What 1,107 accepted papers report for this change
- 765 improved
- 94 no clear effect
- 52 mixed
- 196 hurt
- AudioChat: Unified Audio Storytelling, Editing, and Understanding with Transfusion Forcing ICML 2026 · add structured chain-of-thought reasoning to the diffusion LLM · StoryGen-Eval editing · delta multiFLAM: 27.8 -> 11.7Improved
- Vision Language Models Cannot Reason About Physical Transformation ICML 2026 · add CoT prompt instead of Direct prompt · Number & Length · accuracy: F(3, 333) = 18.28Hurt
- VideoSEG-O3: A Multi-turn Reinforcement Learning Framework for Reasoning Video Object Segmentation ICML 2026 · Add CoT cold-start stage on top of SFT · Long-RVOS · J&F: 47.12 -> 51.82Improved
See this change in detailMatched by the wording of each paper's own result, read by AI from the full text. Papers report gains more often than failures, so this shows what gets published, not the chance a change works.
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| apply step-wise rewards online during beam search instead of post hoc scoring (ablation) | PRA · MedQA test | accuracy: 77.6 -> 81.9 | Improved |
| train the reward agent on teacher step labels (ablation) | PRA with search · MedQA test | accuracy: 76.7 -> 81.9 | Improved |
| add retrieval (search) to an untrained reward agent (ablation) | PRA beam search · MedQA test | accuracy: 74.4 -> 76.7 | Improved |
| use untrained reward agent in beam search without training (ablation) | Qwen3-4B-Instruct policy · MedQA test | accuracy: 74.8 -> 74.4 | No clear effect |
| use outcome-level reward (last step) instead of online process rewards (ablation) | trained PRA · MedQA test | accuracy: 81.9 -> 75.7 | Hurt |
| aggregate process rewards with min post hoc (ablation) | trained PRA · MedQA test | accuracy: 74.8 -> 74.3 | Hurt |
| add PRA guidance versus Self-Consistency across unseen policy models | frozen policies 0.5B to 8B · MedQA test | accuracy: Qwen2.5-0.5B: 28.4 -> 54.1 | Improved |
| increase number of sampled traces from 8 to 64 (ablation) | Self-Consistency · MedQA test | accuracy | No clear effect |
| increase number of sampled traces (inference-time scaling) (ablation) | PRA · MedQA test | accuracy | Improved |
| add Self-Consistency to CoT on hard benchmarks | Qwen3-4B-Instruct · GPQA | accuracy: 60.8 -> 51.8 | Hurt |
| retrieve selectively using learned search score instead of at every step (ablation) | PRA · MedQA test | accuracy | Mixed |
Closest papers by meaning
- Rewarding Progress: Scaling Automated Process Verifiers for LLM Reasoning (ICLR 2025 · similarity 0.90)
- ReasonFlux-PRM: Trajectory-Aware PRMs for Long Chain-of-Thought Reasoning in LLMs (NeurIPS 2025 · similarity 0.89)
- From Mathematical Reasoning to Code: Generalization of Process Reward Models in Test-Time Scaling (AAAI 2026 · similarity 0.88)
- Linking Process to Outcome: Conditional Reward Modeling for LLM Reasoning (ICLR 2026 · similarity 0.88)
- TaTToo: Tool-Grounded Thinking PRM for Test-Time Scaling in Tabular Reasoning (ICLR 2026 · similarity 0.88)
- Rethinking Reward Models for Multi-Domain Test-Time Scaling (NeurIPS 2026 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
