Research map / Large language models / Reasoning and chain of thought

ReVeal: Self-Evolving Code Agents via Reliable Self-Verification

ICLR 2026 · Large language models › Reasoning and chain of thought · approach: RL with verifiable rewards for reasoning · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes ReVeal, a multi-turn RL framework where a single code LLM generates code and self-written tests executed via a Python tool, trained with TAPO (turn-aware policy optimization using joint outcome, generation, and verification rewards) to improve self-verification and test-time scaling on LiveCodeBench.

Problem

Outcome-only RLVR yields unreliable self-verification and limited test-time scaling for code generation, and prior verification approaches rely on separate critics or pre-existing public tests.

Approach

Train a single policy to alternate code generation and self-generated test construction with tool execution feedback, using dense turn-level rewards (generation improvement and verification pass rate against golden code) and a turn-aware advantage (TAPO) that replaces GAE within PPO to assign credit per turn and prevent reward hacking.

Key results

• On LiveCodeBench V6, ReVeal x25 turns reaches 38.7 Pass@1 versus 32.8 for single-turn RL and 31.1 for DAPO-Qwen-32B.
• Pass@1 on LCB V6 rises from 34.8% at turn 1 to 38.7% at turn 25 for ReVeal, despite training with max 3 turns.
• On Qwen2.5-32B-Instruct (LCB V5), TAPO with joint rewards reaches 38.0 Pass@1 vs 33.9 for single-turn RL, with 3.41 delta-up and 0.0 delta-down.

Surprising findings

• Despite training on at most 3 turns, performance keeps improving up to 25 inference turns.
• Prompt-only ReVeal without RL degrades to around 27 Pass@1 under multi-turn settings, well below the RL-trained variant.

Limitations

• Evaluated only on code generation tasks; generality of TAPO to other domains is untested.
• Training verification reward requires golden solutions and filtered test cases, which may not exist in real settings.
• Relies on the Code Judge execution environment and large 32B backbones.

Experimental setup

Datasets: TACO LiveCodeBench CodeContests

Benchmarks: LiveCodeBench V6 LiveCodeBench V5 CodeContests

Metrics: Pass@1 Pass@k delta up (correction rate) delta down (degradation rate)

Baselines: DAPO-Qwen-32B Qwen2.5-32B-Instruct Qwen2.5-Coder-32B-Instruct CTRL Single-turn RL ReVeal w/ outcome reward prompt-only ReVeal

Builds on: PPO GAE DAPO CTRL RLEF verl

Closest papers by meaning

• CURE: Co-Evolving Coders and Unit Testers via Reinforcement Learning (NeurIPS 2025 · similarity 0.90)
• Trust, But Verify: A Self-Verification Approach to Reinforcement Learning with Verifiable Rewards (NeurIPS 2025 · similarity 0.90)
• Learning to Self-Verify Makes Language Models Better Reasoners (ICML 2026 · similarity 0.89)
• CodeScaler: Scaling Code LLM Training and Test-Time Inference via Reward Models (NeurIPS 2026 · similarity 0.88)
• Self-Challenging Language Model Agents (NeurIPS 2025 · similarity 0.88)
• EvoCodeBench: Evaluating Coding Agents in Multi-Turn Iterative Interactions (NeurIPS 2026 · similarity 0.88)