Research map / Large language models / Reasoning and chain of thought
QuRL: Rubrics As Judge For Open-Ended Question Answering
Xiyu Wei, Qingwei Zong, Xiaoguang Li, Eugene J. Yu, Sujian Li
Read the paperCheck your idea against 137,217 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes QuRL, an RL approach for open-ended question answering that uses rubrics as a judge to supply reward signals in place of verifiable ground truth.
Problem
RLVR works well for tasks with gold answers (code, math) but open-ended QA lacks reliable evaluation and verifiable reward signals.
Approach
Uses rubrics as the judge for open-ended QA so that RL can be applied where no single verifiable answer exists; the abstract only indicates rubric-based rewards replace gold-truth verification.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| GRPO with case-wise rubric reward instead of SFT on R1 responses | Qwen2.5-7B · Average of 3 benchmarks | normalized score: 42.3 -> 59.3 | Improved |
| use case-wise rubric reward instead of RLHF scalar reward model (ablation) | QuRL · Average of 3 benchmarks | normalized score: 47.7 -> 59.3 | Improved |
| add case-wise rubrics (vs five-dimension prompt reward) (ablation) | QuRL · Average of 3 benchmarks | normalized score: 44.0 -> 59.3 | Improved |
| add rubric filter (ablation) | QuRL · Average of 3 benchmarks | normalized score: 52.2 -> 59.3 | Improved |
| add web-derived meta-descriptions to rubric generation (ablation) | QuRL · Average of 3 benchmarks | normalized score: 48.9 -> 59.3 | Improved |
| use fixed-maximum rubric scoring instead of free-form (ablation) | QuRL · Average of 3 benchmarks | normalized score: 50.4 -> 59.3 | Improved |
| use fixed-maximum scoring instead of judge-based binary scoring (ablation) | QuRL · Average of 3 benchmarks | normalized score: 52.4 -> 59.3 | Improved |
| score with QuRL rubrics instead of plain LLM-as-a-Judge | GPT-4o judge · Human consistency | Spearman rho: 0.08 -> 0.31 | Improved |
| give LLM judges web articles as references | GPT-4o and DeepSeek-R1 judges · 50 sampled questions | Spearman rho vs human: 0.139 -> 0.209 (GPT-4o w/ web articles); 0.081 -> 0.170 (DeepSeek-R1) | Improved |
| add QuRL rubrics for GPT-4o judge | GPT-4o judge · 50 sampled questions | Spearman rho vs human: 0.139 -> 0.260 | Improved |
Experimental setup
Builds on: RLVR
Closest papers by meaning
- Step-wise Rubric Rewards for LLM Reasoning (NeurIPS 2026 · similarity 0.90)
- Breaking the Exploration Bottleneck: Rubric-Scaffolded Reinforcement Learning for Open-Ended LLM Reasoning (ICML 2026 · similarity 0.89)
- Self-Aligned Reward: Towards Effective and Efficient Reasoners (ICLR 2026 · similarity 0.88)
- RuCL: Stratified Rubric-Based Curriculum Learning for Multimodal Large Language Model Reasoning (ICML 2026 · similarity 0.87)
- Reward Hacking in Rubric-Based Reinforcement Learning (NeurIPS 2026 · similarity 0.87)
- Reinforcing General Reasoning Without Verifiers (ICLR 2026 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
