AI Paper Atlas

Research map / Large language models / Reasoning and chain of thought

DELTA-Code: How RL Unlocks and Transfers New Programming Algorithms in LLMs

Yiyou Sun, Yuhan Cao, Pohao Huang, Haoyue Bai, Hannaneh Hajishirzi, Nouha Dziri, Dawn Song

ICLR 2026 · Large language models › Reasoning and chain of thought · approach: Efficient and adaptive thinking

Read the paperCheck your idea against 137,217 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces DELTA-Code, a controlled benchmark of synthetic coding problem families to test whether RL can teach LLMs genuinely new programming algorithms and transfer them.

Problem

Whether LLMs can acquire or generalize genuinely new reasoning strategies in coding beyond skills already encoded during pre-training or post-training.

Approach

A controlled benchmark of synthetic coding problem families designed to probe learnability and transferability of algorithmic reasoning, measured distributionally, to separate new strategy acquisition from sharpening of existing skills.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
train with binary full-pass reward directly on a pass@K=0 family (ablation)GRPO · Manufactoria-HASfull pass rateHurt
use per-test pass-rate reward instead of full-pass (ablation)GRPO · Manufactoria-HASfull pass rate: full-pass remains <0.01%No clear effect
add per-test pass-rate warm-up then switch to full-pass reward (ablation)GRPO · Manufactoria-HASfull pass rate: 0% -> 100%Improved
add experience replay of successful traces (ablation)GRPO · Manufactoria-HASgrokking time / convergenceMixed
add feedback-in-the-loop (verifier feedback replaces EOS) (ablation)GRPO · Manufactoria-HASgrokking time / stabilityMixed
curriculum via REGEX family before HAS (ablation)GRPO curriculum · Manufactoria-HASpass rateImproved
curriculum via COMPR family before HAS (ablation)GRPO curriculum · Manufactoria-HASpass rateNo clear effect
per-test warm-up on harder PREPEND family (ablation)GRPO · Manufactoria-PREPEND (MEDIUM)full pass rate: stays at 0No clear effect
RL on Basic BouncingSim mixture, test on unseen skill combinationsGRPO · BouncingSim compositionalfull pass rate: near-zero -> 60-70%Improved
RL on Basic BouncingSim, test on Hard tierGRPO · BouncingSim explorative Hardfull pass rate: single digitsNo clear effect
RL on BouncingSim, test on transformative periodic trajectoriesGRPO · BouncingSim transformativefull pass rate: near zeroNo clear effect

Experimental setup

Benchmarks: DELTA-Code

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy