Research map / Large language models / Reasoning and chain of thought

Learning to Generate Unit Test via Adversarial Reinforcement Learning

ICLR 2026 · Large language models › Reasoning and chain of thought · approach: RL with verifiable rewards for reasoning · 2 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes UTRL, an adversarial RL framework that alternately trains a unit test generator (reward: discrimination of LLM code from ground-truth code plus test validity) and a code generator (reward: pass rate on generated tests), using only instruction-code pairs.

Problem

Training LLMs to generate high-quality unit tests usually requires costly ground-truth unit test labels; a reward for test quality without such labels is hard to define.

Approach

Adversarial co-training: the test generator is rewarded for tests that fail LLM-generated code yet pass ground-truth code (discrimination reward), and the code generator is rewarded for passing the generated tests, so only instruction-code solution pairs are needed.

Key results

• Qwen3-4B trained with UTRL reaches 14.9% best-of-32 code accuracy with Qwen3-8B code generator vs 11.7% for SFT on ground-truth tests and 13.3% for GPT-4.1.
• Qwen3-14B + UTRL achieves unit test fidelity 0.827 (Qwen3-4B + UTRL 0.794), with Qwen3-14B outperforming GPT-4.1 on fidelity.
• Code generator trained with UTRL reaches 15.3% pass@1 on TACO, close to 15.9% from training on GT unit tests, versus 3.6% for SFT on ground-truth code.

Surprising findings

• SFT on ground-truth unit tests yields only marginal gains in unit test fidelity despite large best-of-N gains.
• Code generator SFT on ground-truth solutions severely degraded performance on unseen tasks (3.6% pass@1).

Limitations

• Gains saturate after the second adversarial iteration; third iteration showed no notable improvement with Qwen3-4B.
• Performance depends on balancing discrimination and validity rewards (lambda); lambda=0 or 1 degrades quality.
• Evaluated mainly on competitive programming in Python with stdio-format tests.

Experimental setup

Datasets: TACO CodeContests LiveCodeBench-v2

Benchmarks: TACO evaluation (945 tasks) LiveCodeBench-v2 (511 tasks)

Metrics: best-of-N code score best-of-N code accuracy unit test fidelity (Spearman correlation) pass@1 code accuracy validity ratio

Baselines: Qwen3-4B Qwen3-8B Qwen3-14B Qwen3-32B Deepseek-Coder-V2-Lite GPT-4o GPT-4.1 SFT w/ D_UT SFT w/ D_reason+UT CURE Ground-truth unit tests

Builds on: GRPO CURE CodeRM UTGEN

Closest papers by meaning

• CURE: Co-Evolving Coders and Unit Testers via Reinforcement Learning (NeurIPS 2025 · similarity 0.90)
• Rethinking Verification for LLM Code Generation: From Generation to Testing (NeurIPS 2025 · similarity 0.90)
• TTRL: Test-Time Reinforcement Learning (NeurIPS 2025 · similarity 0.89)
• CodeScaler: Scaling Code LLM Training and Test-Time Inference via Reward Models (NeurIPS 2026 · similarity 0.88)
• Teaching Language Models to Critique via Reinforcement Learning (ICML 2025 · similarity 0.88)
• Compute Aligned Training: Optimizing for Test Time Inference (NeurIPS 2026 · similarity 0.87)