Research map / Large language models / Reasoning and chain of thought
Themis: Automated Constraint-Aware Test Synthesis Framework for Code Reinforcement Learning
Shengyu Ye, Qi Liu, Hao Jiang, Zheng Zhang, Heng Yu, Zhenya Huang
Read the paperCheck your idea against 137,217 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Themis, a constraint-aware automated test synthesis framework that generates higher-quality test suites to provide more reliable reward signals for RL-based code generation training of LLMs.
Problem
RL for code generation in LLMs depends on high-quality test suites for reward signals, but current test suites have insufficient quantity and quality, causing false positives (incorrect solutions passing) and slow positives (valid but suboptimal solutions passing slowly).
Approach
Constraint-aware automated synthesis of test cases for code RL, aimed at reducing both false positives and slow positives in verification rewards. Details beyond the abstract are not available.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| RL with Themis-augmented tests instead of original TACO tests | Qwen2.5-Coder-7B-Instruct · Average of LCB, HumanEval, MBPP | Pass@1 (%): 57.9 -> 59.4 | Improved |
| RL with Themis tests instead of no RL | Qwen2.5-7B · Average of LCB, HumanEval, MBPP | Pass@1 (%): 41.1 -> 56.7 | Improved |
| RL with Themis tests vs original TACO tests | Qwen2.5-7B-Instruct · Average of LCB, HumanEval, MBPP | Pass@1 (%): 55.3 -> 57.1 | Improved |
| RL with original TACO tests on already strong instruct model | Qwen2.5-Coder-7B-Instruct · Average of LCB, HumanEval, MBPP | Pass@1 (%): 57.2 -> 57.9 | No clear effect |
| training on random KodCode 10k (false-positive-prone tests) (ablation) | Qwen2.5-Coder-7B-Instruct · Average of LCB, HumanEval, MBPP | Pass@1 (%): 57.2 -> 56.7 | Hurt |
| filter KodCode samples with few tests or low difficulty (ablation) | Qwen2.5-Coder-7B-Instruct · Average of LCB, HumanEval, MBPP | Pass@1 (%): 56.7 -> 57.7 | Improved |
| augment filtered KodCode with Themis tests (ablation) | Qwen2.5-Coder-7B-Instruct · Average of LCB, HumanEval, MBPP | Pass@1 (%): 57.7 -> 58.8 | Improved |
| Themis test reduction (about 5x compression) (ablation) | Themis tests · TACO-verified error detection (DeepSeek-R1 generator) | error detection rate (%): 95.6 -> 94.7 | No clear effect |
| Themis tests vs EvalPlus-style tests | Test generation · TACO-verified error detection (DeepSeek-R1 generator) | error detection rate (%): 66.2 -> 95.6 | Improved |
| Themis tests vs original TACO tests | Test generation · TACO-verified error detection (DeepSeek-R1 generator) | error detection rate (%): 89.3 -> 95.6 | Improved |
Closest papers by meaning
- Rethinking Verification for LLM Code Generation: From Generation to Testing (NeurIPS 2025 · similarity 0.90)
- CodeScaler: Scaling Code LLM Training and Test-Time Inference via Reward Models (NeurIPS 2026 · similarity 0.90)
- CURE: Co-Evolving Coders and Unit Testers via Reinforcement Learning (NeurIPS 2025 · similarity 0.90)
- RLEF: Grounding Code LLMs in Execution Feedback with Reinforcement Learning (ICML 2025 · similarity 0.89)
- Afterburner: Reinforcement Learning Facilitates Self-Improving Code Efficiency Optimization (NeurIPS 2025 · similarity 0.89)
- Learning to Generate Unit Test via Adversarial Reinforcement Learning (ICLR 2026 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
