Research map / Large language models / Reasoning and chain of thought

StepCodeReasoner: Aligning Code Reasoning with Stepwise Execution Traces via Reinforcement Learning

ICML 2026 · Large language models › Reasoning and chain of thought · approach: Test-time compute scaling · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes StepCodeReasoner, which inserts print-based execution-trace anchors into Python code and trains a Qwen2.5-Coder-7B model with SFT then Bi-Level GRPO (group-relative plus intra-trajectory stepwise rewards) to predict intermediate states and outputs for code reasoning.

Problem

Code reasoning methods supervise only final outputs, so models can reach correct answers through inconsistent intermediate execution reasoning (reward hacking / right answer, wrong logic).

Approach

Insert deterministic print-based execution anchors via a teacher LLM so each intermediate state is verifiable against an interpreter, and combine group-relative stepwise advantages with an intra-trajectory shaping term that credits steps by downstream correctness.

Key results

• StepCodeReasoner-7B reaches 91.1% on CRUXEval and 86.5% on LiveCodeBench, vs CodeReasoner-7B 86.0% and 77.7% and GPT-4o 85.6% and 75.1%.
• On REval, StepCodeReasoner-7B scores 82.9% average vs CodeReasoner-14B 81.1% and GPT-4o 77.3%.
• Step accuracy rises from 63.5% (CodeReasoner-7B) to 80.7% with final accuracy 91.6% on CRUXEval-O.

Surprising findings

• Even GPT-4o achieves only about 73% accuracy judging intermediate execution states as a PRM, motivating rule-based rewards.
• The 7B model without instrumentation at inference still performs competitively (CRUXEval 0.848), while the main results use instrumented prompts with extra tokens.

Limitations

• Print-anchor instrumentation is Python-specific and relies on observable print output; not applicable to compiled languages or side-effecting programs.
• Anchor rules (no prints in loops, 10-line trace cap) may miss fine-grained state changes in nested loops or recursion.
• Inference-time instrumentation increases token cost by about 1.5x.

Experimental setup

Datasets: CodeReasoner training data CRUXEval LiveCodeBench REval HumanEval MBPP

Benchmarks: CRUXEval LiveCodeBench REval HumanEval MBPP

Metrics: pass@1 accuracy step accuracy

Baselines: GPT-4o GPT-4o-mini Qwen2.5-72B-Instruct Llama3-70B-Instruct Qwen2.5-Coder-32B-Instruct Qwen3-Coder-30B-A3B-Instruct SemCoder CodeI/O CodeReasoner

Builds on: GRPO CodeReasoner Step-GRPO

Closest papers by meaning

• Chain of Execution Supervision Promotes General Reasoning in Large Language Models (NeurIPS 2025 · similarity 0.88)
• Reasoning Through Execution: Unifying Process and Outcome Rewards for Code Generation (ICML 2025 · similarity 0.88)
• MURPHY: Feedback-Aware GRPO with Retrospective Credit Assignment for Multi-Turn Code Generation (NeurIPS 2026 · similarity 0.87)
• CodeScaler: Scaling Code LLM Training and Test-Time Inference via Reward Models (NeurIPS 2026 · similarity 0.86)
• Afterburner: Reinforcement Learning Facilitates Self-Improving Code Efficiency Optimization (NeurIPS 2025 · similarity 0.86)
• rStar-Coder: Scaling Competitive Code Reasoning with a Large-Scale Verified Dataset (NeurIPS 2025 · similarity 0.86)