Research map / Reinforcement learning / RL from feedback
rePIRL: Learn PRM with Inverse RL for LLM Reasoning
Xian Wu, Kaijie Zhu, Ying Zhang, Lun Wang, Wenbo Guo
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| learn PRM with inverse RL (rePIRL) instead of vanilla outcome-reward RLOO | Qwen2.5-3B-Instruct · Math average | accuracy: 31.7 -> 33.5 | Improved |
| learn PRM with inverse RL (rePIRL) instead of vanilla RLOO | Qwen3-4B-Base · Coding average | accuracy: 35.5 -> 39.8 | Improved |
| use rePIRL PRM versus PRIME online PRM | Qwen3-4B-Base · Math average | accuracy: 39.3 -> 41.6 | Improved |
| add SFT warm-up on expert trajectories before RLOO | Qwen2.5-3B-Instruct · Math average | accuracy: 31.7 -> 25.7 (RLOO vs SFT+RLOO) | Hurt |
| offline methods (BC, KTO) instead of online RL | Qwen2.5-3B-Instruct · Math average | accuracy: 28.4 -> 20.6 (BC) | Hurt |
| use rePIRL PRM alone (no outcome reward) to train policy from scratch on MATH | Qwen3-4B-Base · Test-time training avg | accuracy: 42.3 (RLOO) vs 42.1 (rePIRL) | No clear effect |
| use PRM to select responses instead of majority voting | Qwen3-4B-Base · AIME-2024 | accuracy: 0.200 -> 0.230 (majority voting vs rePIRL at 64 samples) | Improved |
| train on hard problems with rePIRL PRM instead of outcome reward | Qwen3-4B-Base · Hard MATH subset | accuracy | Improved |
| remove outcome reward, train with PRM only (ablation) | rePIRL · MATH-500 | accuracy: 46.0 -> 59.6 (vs base model) | Improved |
| add corrected policy trajectories as additional expert data for reward model (ablation) | rePIRL · training stability | stability and performance | Improved |
| rePIRL vs RLOO on MATH500 with Qwen2.5-3B-Instruct | Qwen2.5-3B-Instruct · MATH500 | pass@1: 63.6 -> 62.4 | No clear effect |
| rePIRL vs PRIME on LiveCodeBench | Qwen3-4B-Base · LiveCodeBench | pass@1: 19.0 -> 27.5 | Improved |
Closest papers by meaning
- Tricks or Traps? A Deep Dive into RL for LLM Reasoning (ICLR 2026 · similarity 0.89)
- ReST-RL: Reinforcing LLM Reasoning through Unified Self-Training and Value-Guided Search (NeurIPS 2026 · similarity 0.89)
- Is PRM Necessary? Problem-Solving RL Implicitly Induces PRM Capability in LLMs (NeurIPS 2025 · similarity 0.88)
- Is Inverse Reinforcement Learning Harder than Standard Reinforcement Learning? A Theoretical Perspective (ICML 2024 · similarity 0.88)
- Rectifying LLM Thought from Lens of Optimization (ICLR 2026 · similarity 0.88)
- The Surprising Effectiveness of Negative Reinforcement in LLM Reasoning (NeurIPS 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
