Research map / Reinforcement learning / RL from feedback
Rubrics as Rewards: Reinforcement Learning Beyond Verifiable Domains
Anisha Gunjal, Anthony Wang, Elaine Lau, Vaskar Nath, Yunzhong He, Bing Liu, Sean M. Hendryx
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace direct Likert reward with rubric-based implicit reward | GRPO on Qwen2.5-7B · HealthBench | overall score: up to 31% relative gain | Improved |
| replace direct Likert reward with rubric-based implicit reward | GRPO on Qwen2.5-7B · GPQA-Diamond | accuracy: up to 7% relative gain | Improved |
| replace reference-Likert reward with rubric reward | GRPO on Qwen2.5-7B · HealthBench-1k | overall score: 31.7% -> 35.9% | Improved |
| use generic fixed rubrics instead of instance-specific rubrics (RaR-Predefined) (ablation) | RaR · HealthBench / GPQA-Diamond | score | Hurt |
| generate rubrics with access to reference answers (ablation) | RaR-Implicit-Synthetic · HealthBench-1k | overall score: 32.0% -> 35.9% | Improved |
| add reference answers vs human-authored rubrics (ablation) | RaR-Implicit · HealthBench-1k | overall score: 34.8% (human) vs 35.9% (synthetic with ref) | No clear effect |
| use all rubric criteria instead of Essential-only (ablation) | RaR · HealthBench-1k | overall score: 34.9% -> 37.2% | Improved |
| add pitfall criteria (ablation) | RaR · HealthBench-1k | overall score: 37.2% -> 37.2% | No clear effect |
| add categorical weight labels (ablation) | RaR · HealthBench-1k | overall score: 38.8% (none) vs 37.2% (labels) | No clear effect |
| use a stronger LLM to generate reference-free rubrics (ablation) | RaR · HealthBench-1k | overall score: GPT-4o 34.2%, Qwen-32B 31.1%, Qwen-7B 31.9% | Mixed |
| give judge rubrics (RaR-Implicit) instead of direct Likert | Qwen-7B-Instruct judge · synthetic medical judge quality | judge accuracy: 22.0% -> 26.7% | Improved |
| give judge rubrics instead of direct Likert | Qwen-14B-Instruct judge · synthetic medical judge quality | judge accuracy: 24.9% -> 25.0% | No clear effect |
Closest papers by meaning
- Reward Hacking in Rubric-Based Reinforcement Learning (NeurIPS 2026 · similarity 0.94)
- Not Every Rubric Teaches Equally: Policy-Aware Rubric Rewards for RLVR (NeurIPS 2026 · similarity 0.94)
- Step-wise Rubric Rewards for LLM Reasoning (NeurIPS 2026 · similarity 0.93)
- RM-R1: Reward Modeling as Reasoning (ICLR 2026 · similarity 0.91)
- Deep Research as Rubric (NeurIPS 2026 · similarity 0.90)
- From Verifiable Dot to Reward Chain: Harnessing Verifiable Reference-based Rewards for Reinforcement Learning of Open-ended Generation (ICLR 2026 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
