Research map / Large language models / Agents, code and math
CodeRL: Mastering Code Generation through Pretrained Models and Deep Reinforcement Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Apply CodeRL actor-critic training to CodeT5 (ablation) | CodeT5-770M on APPS · APPS test (All) | pass@1: 1.50 -> 2.20 | Helped |
| Combine cross-entropy and RL losses instead of using either alone (ablation) | CodeT5-770M · APPS test (All) | pass@5: 2.26 -> 3.10 | Helped |
| Use critic token-level returns with a greedy baseline rather than identical per-token sample rewards (ablation) | CodeT5-770M · APPS test (All) | pass@1: 1.62 -> 2.20 | Helped |
| Remove the greedy baseline from critic-based return estimation (ablation) | CodeT5-770M · APPS test (All) | pass@1: 2.20 -> 1.36 | Hurt |
| Replace learned critic token values with linearly decaying position-based values (ablation) | CodeT5-770M · APPS test (All) | pass@1: 2.20 -> 1.64 | Hurt |
| Add program refining to inference (ablation) | CodeT5 on APPS · APPS test (All) | pass@1000: 17.78 -> 19.36 | Helped |
| Add program repair of top failed candidates alongside refining (ablation) | CodeT5 on APPS · APPS test (All) | pass@1000: 19.36 -> 20.98 | Helped |
| Increase the number of repair candidates from M=1 to M=4 (ablation) | Critic Sampling with CodeT5 · APPS test (All) | pass@1000: 20.98 -> 20.62 | Hurt |
| Add next-token prediction pretraining to MSP with GCPY data (ablation) | CodeT5-770M · APPS test (All) | pass@5: 2.06 -> 2.90 | Helped |
| Increase pretraining data by adding GCPY to CSN (ablation) | CodeT5-770M · APPS test (All) | pass@1: 1.30 -> 1.56 | Helped |
| Increase CodeT5 model size from 60M to 770M (ablation) | CodeT5 pretrained on CSN with MSP · APPS test (All) | pass@1: 0.68 -> 1.30 | Helped |
| Use CodeRL rather than the APPS-finetuned CodeT5 model alone (ablation) | CodeT5-770M, zero-shot transfer · MBPP test | pass@80: 60.2 -> 63.0 | Helped |
Closest papers by meaning
- $\mathcal{B}$-Coder: Value-Based Deep Reinforcement Learning for Program Synthesis (ICLR 2024 · similarity 0.91)
- RLEF: Grounding Code LLMs in Execution Feedback with Reinforcement Learning (ICML 2025 · similarity 0.90)
- Teaching Language Models to Critique via Reinforcement Learning (ICML 2025 · similarity 0.90)
- CodeScaler: Scaling Code LLM Training and Test-Time Inference via Reward Models (NeurIPS 2026 · similarity 0.89)
- ESRL: Efficient Sampling-Based Reinforcement Learning for Sequence Generation (AAAI 2024 · similarity 0.89)
- Generating Code World Models with Large Language Models Guided by Monte Carlo Tree Search (NeurIPS 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.