Research map / Large language models / Reasoning and chain of thought

Deep Research as Rubric

NeurIPS 2026 · Large language models › Reasoning and chain of thought · approach: RL with verifiable rewards for reasoning

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes DR-Rubric, which builds task-specific atomic rubrics via multi-turn deep-research evidence gathering and uses them as GRPO rewards, including bootstrap rubric generation where the trained 8B model writes its own rubrics.

Problem

Open-ended reasoning and long-form generation lack reliable automatic reward signals; existing rubrics are either hand-authored or prompt-generated from parametric priors, missing knowledge-intensive task-specific criteria.

Approach

Treat rubric construction as a research problem: an agentic deep-research stage elicits domain facts, constraints and failure modes, then a synthesis stage distills them into atomic, verifiable constraints used as a mean-of-satisfaction reward for GRPO; the policy can itself act as the rubric generator in bootstrap iterations.

Key results

• DR-Rubric-8B (GPT-5) reaches 73.7 on ResearchQA and 43.0 on DeepResearchBench, +3.8 and +3.2 over the best baseline.
• DR-Rubric-8B (BS-3) with 3K RL instances reaches MMLU-Pro 78.0 (+3.7) and MMLU 85.3 (+1.5), with no external model used in the bootstrap steps.
• Bootstrap polarization signal indicates over-bootstrapping on Qwen3-30B-A3B: agentic score collapses from 51.3 to 22.7 at BS-4.

Surprising findings

• Benchmark-native rubrics are not an upper bound; GPT-5, Gemini and bootstrap rubrics surpass them on multiple benchmarks despite protocol misalignment.
• Bootstrap rubrics move away from GPT-5-style structure, with fewer dimensions and longer constraints, and BS-2 performance dips before BS-3 recovers.

Limitations

• Instability under over-bootstrapping; reward hacking and collapse beyond the recommended 2-3 steps.
• Evaluation uses a shared mock retrieval backend rather than live web search in the main results.
• Limited large-scale validation; results rely on single-run comparisons with small gaps on some benchmarks.

Experimental setup

Datasets: ResearchQA RaR-Science HealthBench RaR-Medicine long-form writing

Benchmarks: ResearchQA DeepResearchBench LocalSearchBench GPQA MMLU-Pro MMLU

Metrics: rubric F1 RACE+FACT correctness accuracy

Baselines: Qwen2.5-7B Qwen3-8B Qwen3-8B-SFT Search-R1-7B DR-Tulu-SFT-8B DR-Tulu-RL-8B WebExplorer-8B DeepSeek-R1-Distill-Qwen-14B WebThinker-R1-14B Tongyi-DeepResearch-30B-A3B

Builds on: GRPO ReAct DeerFlow DR-Tulu

Closest papers by meaning

• Not Every Rubric Teaches Equally: Policy-Aware Rubric Rewards for RLVR (NeurIPS 2026 · similarity 0.90)
• Reward Hacking in Rubric-Based Reinforcement Learning (NeurIPS 2026 · similarity 0.89)
• Step-wise Rubric Rewards for LLM Reasoning (NeurIPS 2026 · similarity 0.88)
• Auto-Rubric as Reward: From Implicit Preference to Explicit Generative Criteria (NeurIPS 2026 · similarity 0.86)
• Breaking the Exploration Bottleneck: Rubric-Scaffolded Reinforcement Learning for Open-Ended LLM Reasoning (ICML 2026 · similarity 0.86)
• DeepSearch: Overcome the Bottleneck of Reinforcement Learning with Verifiable Rewards via Monte Carlo Tree Search (ICLR 2026 · similarity 0.86)