Research map / Reinforcement learning / RL from feedback
Reward and Guidance through Rubrics: Promoting Exploration to Improve Multi-Domain Reasoning
Baolong Bi, Shenghua Liu, Yiwei Wang, Siqian Tong, Lingrui Mei, Yuyao Ge, Yilong Xu, Jiafeng Guo, Xueqi Cheng
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use rubric reward (factual + process) instead of outcome-only reward | GRPO · multi-domain average, Qwen2.5-7B | average accuracy (%): 52.3 -> 53.7 | Improved |
| use Likert reward instead of outcome-only reward | GRPO · multi-domain average, Qwen2.5-7B | average accuracy (%): 52.3 -> 52.7 | Improved |
| add rubric-guided off-policy refinement to Rubric-GRPO | Rubric-GRPO · multi-domain average, Qwen2.5-7B | average accuracy (%): 53.7 -> 55.8 | Improved |
| add rubric-guided off-policy refinement to Rubric-GRPO | Rubric-GRPO · multi-domain average, Qwen2.5-3B | average accuracy (%): 43.6 -> 45.5 | Improved |
| add Exploration Assessment (EA) (ablation) | RGR-GRPO · multi-domain average, Qwen2.5-7B | average accuracy (%): 53.8 -> 55.8 | Improved |
| add off-policy shaping function (ablation) | RGR-GRPO · multi-domain average, Qwen2.5-7B | average accuracy (%): 54.5 -> 55.8 | Improved |
| add process criteria to factual criteria in rubrics (ablation) | RGR-GRPO · multi-domain average, Qwen2.5-7B | average accuracy (%): 55.2 -> 55.8 | Improved |
| add process criteria to factual criteria in on-policy rubric reward (ablation) | Rubric-GRPO · multi-domain average, Qwen2.5-7B | average accuracy (%): 53.5 -> 53.7 | No clear effect |
| use off-policy baselines Critique-GRPO or LUFFY instead of on-policy GRPO | GRPO · multi-domain average, Qwen2.5-7B | average accuracy (%): 52.3 -> 50.6 (Critique), 50.5 (LUFFY) | Hurt |
| add rubric-guided self-refinement at test time (ablation) | Outcome-GRPO checkpoints · WebInstruct-Verify 4K held-out | outcome score: gain of +16.9 to +18.5 across stages | Improved |
| add rubric-guided off-policy guidance for pass@k | RGR-GRPO · SciBench Math pass@k (Qwen2.5-7B) | pass@k | Improved |
Closest papers by meaning
- Step-wise Rubric Rewards for LLM Reasoning (NeurIPS 2026 · similarity 0.92)
- RM-R1: Reward Modeling as Reasoning (ICLR 2026 · similarity 0.91)
- Breaking the Exploration Bottleneck: Rubric-Scaffolded Reinforcement Learning for Open-Ended LLM Reasoning (ICML 2026 · similarity 0.91)
- Reinforcement Learning for Reasoning in Large Language Models with One Training Example (NeurIPS 2025 · similarity 0.90)
- Revisiting Reinforcement Learning for LLM Reasoning from A Cross-Domain Perspective (NeurIPS 2025 · similarity 0.90)
- Reinforcement Learning with Verifiable Rewards Implicitly Incentivizes Correct Reasoning in Base LLMs (ICLR 2026 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
