Research map / Reinforcement learning / RL from feedback
Unlocking Token Rewards via Training-Free Reward Attribution
Sitong Wu, Haoru Tan, Bin Xia, Xichen Zhang, Jingyao Li, Shaofeng Zhang, Xiaojuan Qi, Bei Yu, Jiaya Jia
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add P2T token-level reward to GRPO | Qwen2.5-Math-7B · 7 math benchmarks avg | Pass@1: 48.0 -> 53.8 | Improved |
| add P2T token-level reward to GRPO | LLaMA3.2-3B-Instruct · 7 math benchmarks avg | Pass@1: 26.4 -> 35.8 | Improved |
| add P2T token-level reward to GRPO | DeepSeek-R1-Distill-Qwen-1.5B · 7 math benchmarks avg | Pass@1: 50.4 -> 57.2 | Improved |
| add P2T token-level reward to GRPO | Qwen3-1.7B (non-thinking mode) · 7 math benchmarks avg | Pass@1: 47.8 -> 53.0 | Improved |
| add P2T token reward versus PRIME and SPRO token rewards | Eurus-2-7B-SFT · Eurus-2 math and code avg | Pass@1: 38.4 -> 40.7 (SPRO), 36.0 -> 40.7 (PRIME) | Improved |
| add token-level redistribution of step reward within each step (vs step reward only) (ablation) | GRPO + PRM reward, Qwen2.5-Math-7B · 7 math benchmarks avg | Pass@1: 50.6 -> 53.8 | Improved |
| add step-level process reward to outcome reward (ablation) | GRPO, Qwen2.5-Math-7B · 7 math benchmarks avg | Pass@1: 48.0 -> 50.6 | Improved |
| vary redistribution strength eta from 0.25 to 1.0 (ablation) | P2T · 7 math benchmarks avg | Pass@1: 53.5 / 53.8 / 53.4 / 53.2 | No clear effect |
| use signed redistribution instead of magnitude-based (ablation) | P2T · 7 math benchmarks avg | Pass@1: 53.8 -> 52.6 | Hurt |
| use stronger PRM (ReasonFlux-PRM-7B vs Skywork PRM-7B) (ablation) | P2T text-only · 7 math benchmarks avg | Pass@1: 50.0 -> 53.8 | Improved |
| use VisualPRM instead of MM-PRM (ablation) | P2T Qwen2.5-VL-7B-Instruct · 5 multimodal benchmarks avg | Pass@1: 55.3 -> 57.5 | Improved |
| use pad token as null token instead of EOS token (ablation) | P2T · 7 math benchmarks avg | Pass@1: 50.3 -> 53.8 | Improved |
Closest papers by meaning
- Discriminative Policy Optimization for Token-Level Reward Models (ICML 2025 · similarity 0.89)
- Scaling Reward Modeling without Human Supervision (NeurIPS 2026 · similarity 0.89)
- Training-Trajectory-Aware Token Selection (ICML 2026 · similarity 0.88)
- Rewarding Progress: Scaling Automated Process Verifiers for LLM Reasoning (ICLR 2025 · similarity 0.87)
- Linking Process to Outcome: Conditional Reward Modeling for LLM Reasoning (ICLR 2026 · similarity 0.87)
- Stop Summation: Min-Form Credit Assignment Is All Process Reward Model Needs for Reasoning (NeurIPS 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
