Research map / Large language models / Reasoning and chain of thought

Primal Generation, Dual Judgment: Self-Training from Test-Time Scaling

NeurIPS 2026 · Large language models › Reasoning and chain of thought · approach: Test-time compute scaling · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes DuST, self-training code models with GRPO to rank their own execution-labeled candidates (never rewarding generation directly), which also improves single-sample code generation and best-of-4 test-time scaling.

Problem

Code generation RL uses sparse pass/fail feedback per sample, and the comparative information revealed by generate-then-judge test-time scaling is discarded after inference without improving the generator.

Approach

Train the same model on-policy via GRPO to rank its own execution-labeled mixed-correctness candidate groups with a pairwise correct-over-incorrect ranking reward; the discriminative judgment training transfers back to improve primal code generation, unlike SFT on the same ranking data.

Key results

• Qwen3-30B-Thinking on LCB v6: test-time scaling accuracy 68.7% -> 72.6% (+3.9) and judgment NDCG 70.1 -> 76.3.
• Single-sample generation pass@1 improves 65.4% -> 68.5% (+3.1) on LCB v6 without any direct generation reward.
• SFT on the same ranking data (teacher Qwen3-235B-Thinking) improves judgment (NDCG 70.1 -> 73.6) but leaves pass@1 at 65.4%, while GRPO improves both.

Surprising findings

• Training only on ranking rewards yields stronger code generation (pass@1 68.3%) than training directly on binary execution pass/fail reward (67.1%).
• SFT on teacher rankings improves judgment but does not transfer to generation, whereas on-policy GRPO does.

Limitations

• Relies on execution-grounded correctness, so it is less directly applicable to open-ended generation tasks.
• Effectiveness depends on mixed-quality candidate groups; too-weak or too-strong models yield little signal.
• Test suites may be imperfect, introducing label noise; mechanism of transfer only partially understood.

Experimental setup

Datasets: rSTARcoder seed subset LiveCodeBench v6 LiveCodeBench v5

Benchmarks: LiveCodeBench v6 LiveCodeBench v5

Metrics: Best-of-4 accuracy pass@1 NDCG

Baselines: OpenReasoning-Nemotron-32B TinyR1-32B OpenCodeReasoning-1.1-32B OpenReasoning-Nemotron-14B OpenCodeReasoning-1.1-14B OpenReasoning-Nemotron-7B Klear-Reasoner-8B DeepSeek-R1-0528-Qwen3-8B OpenCodeReasoning-1.1-7B Skywork-OR1-7B

Builds on: GRPO rSTARcoder LiveCodeBench veRL

Closest papers by meaning

• Test-Time Scaling in the Wild: Why Exploitation, Not Exploration, Is the Bottleneck (NeurIPS 2026 · similarity 0.88)
• CodeChemist: Test-Time Scaling for Low-Resource Code Generation via Functional Knowledge Transfer (ICML 2026 · similarity 0.87)
• CodeScaler: Scaling Code LLM Training and Test-Time Inference via Reward Models (NeurIPS 2026 · similarity 0.87)
• From Mathematical Reasoning to Code: Generalization of Process Reward Models in Test-Time Scaling (AAAI 2026 · similarity 0.87)
• Reasoning Through Execution: Unifying Process and Outcome Rewards for Code Generation (ICML 2025 · similarity 0.87)
• V1: Unifying Generation and Self-Verification for Parallel Reasoners (ICML 2026 · similarity 0.86)