Research map / Large language models / AI and society
Training AI Co-Scientists Using Rubric Rewards
Shashwat Goel, RISHI HAZRA, Dulhan Jayalath, Timon Willi, Parag Jain, William F. Shen, Ilias Leontiadis, Francesco Barbieri, Yoram Bachrach, Jonas Geiping, Chenxi Whitehouse
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| train with rubric-based GRPO and 4B self-reward (ablation) | Qwen-3-4B · validation, Claude-4-Sonnet | rubric items satisfied (%): 12.0 -> 21.7 | Improved |
| use SFT on reference solutions instead of RL (ablation) | Qwen-3-4B · validation, Claude-4-Sonnet | rubric items satisfied (%): 12.0 -> 3.4 | Hurt |
| disable the KL penalty (ablation) | Rubric RL with 30B RM · validation, Claude-4-Sonnet | rubric items satisfied (%): 23.3 -> 29.7 | Improved |
| use stronger 30B MoE grader instead of 4B grader (ablation) | Rubric RL with KL penalty · validation, Claude-4-Sonnet | rubric items satisfied (%): 21.7 -> 23.3 | Improved |
| add reference solution for the grader (ablation) | Rubric RL (4B RM) · validation, Claude-4-Sonnet | rubric items satisfied (%): 20.0 -> 21.7 | Improved |
| add goal-specific rubrics to grading (ablation) | Rubric RL 30B RM · validation, Claude-4-Sonnet | rubric items satisfied (%): 20.9 -> 29.7 | Improved |
| add general guidelines to grading (ablation) | Rubric RL 30B RM · validation, Claude-4-Sonnet | rubric items satisfied (%): 19.8 -> 29.7 | Improved |
| add data filtering (ablation) | Rubric RL 30B RM · validation, Claude-4-Sonnet | rubric items satisfied (%): 28.8 -> 29.7 | Improved |
| train with rubric RL and compare to initial model with human experts | Qwen-3-30B-A3B-Instruct · ResearchPlanGen-ML human study | expert overall score (1-10): 7.31 -> 7.89 | Improved |
| apply medical-finetuned model to ArXiv | Qwen-3-30B · ResearchPlanGen-ArXiv | relative improvement: 17.5% | Improved |
| train on Llama-3.1-8B with the same recipe | Llama-3.1-8B · ML research goals | rubric score: 4.9 -> 18.5 | Improved |
| train on Gemma-3-4B with the same recipe | Gemma-3-4B · ML research goals | rubric score: 12.0 -> 16.7 | Improved |
Closest papers by meaning
- DR Tulu: Reinforcement Learning with Evolving Rubrics for Deep Research (ICML 2026 · similarity 0.85)
- Deep Research as Rubric (NeurIPS 2026 · similarity 0.85)
- AI-Researcher: Autonomous Scientific Innovation (NeurIPS 2025 · similarity 0.85)
- AAAR-1.0: Assessing AI’s Potential to Assist Research (ICML 2025 · similarity 0.85)
- Predicting Empirical AI Research Outcomes with Language Models (NeurIPS 2025 · similarity 0.84)
- CycleResearcher: Improving Automated Research via Automated Review (ICLR 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
