Research map / Large language models / Agents, code and math

RLEF: Grounding Code LLMs in Execution Feedback with Reinforcement Learning

ICML 2025 · Large language models › Agents, code and math · approach: Software engineering agents · 6 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes RLEF, an end-to-end PPO-based RL method that trains Llama 3.1 Instruct models to use execution feedback over multi-turn code repair, reaching state-of-the-art CodeContests solve rates at small sample budgets.

Problem

Code LLMs struggle to use execution feedback iteratively; independent sampling often beats iterative repair at fixed inference budgets, and prior agentic scaffolds are costly.

Approach

Frame iterative code synthesis as a partially observable MDP where public-test execution feedback is inserted into the dialog, and train the LLM end-to-end with PPO using a binary reward from private tests, with a token-level policy and turn-level value function.

Key results

• Llama 3.1 70B Instruct with RLEF reaches 40.1 1@3 on CodeContests test set, versus 27.5 before RLEF and 29 for AlphaCodium with GPT-4 at 5@100.
• Llama 3.1 8B Instruct with RLEF improves 1@3 from 10.5 to 16.0 on CodeContests test set.
• With 10@100, the 70B model reaches 54.5 on both valid and test sets after RLEF, up from 50.2 and 50.3.

Measured effects reported in the paper

Surprising findings

• Few-shot prompting is detrimental to instruction-tuned models on CodeContests.
• Withholding public test execution feedback during RL leads to a significantly worse model (12.2 valid, 10.9 test vs 17.2 and 16.0).

Limitations

• Task is limited to improving a single solution to a given problem; no decomposition into larger tasks.
• Requires unit test cases with execution feedback, which may not be available.
• Large compute needed: 288 GPUs for 8B and 2304 GPUs for 70B.

Experimental setup

Datasets: CodeContests HumanEval+ MBPP+

Benchmarks: CodeContests HumanEval+ MBPP+

Metrics: n@k solve rate pass@1 pass@10 chrF

Baselines: AlphaCode AlphaCodium MapCoder Code Llama 34B + PPO gpt-4o-2024-05-13 SFT Few-Shot

Builds on: PPO AlphaCode AlphaCodium CodeContests

Closest papers by meaning

• Afterburner: Reinforcement Learning Facilitates Self-Improving Code Efficiency Optimization (NeurIPS 2025 · similarity 0.90)
• R1-Code-Interpreter: LLMs Reason with Code via Supervised and Multi-stage Reinforcement Learning (ICLR 2026 · similarity 0.90)
• CodeScaler: Scaling Code LLM Training and Test-Time Inference via Reward Models (NeurIPS 2026 · similarity 0.90)
• Teaching Language Models to Critique via Reinforcement Learning (ICML 2025 · similarity 0.90)
• Generalizable End-to-End Tool-Use RL with Synthetic CodeGym (ICLR 2026 · similarity 0.89)
• Generating Code World Models with Large Language Models Guided by Monte Carlo Tree Search (NeurIPS 2024 · similarity 0.89)