Research map / Large language models / Reasoning and chain of thought

Curriculum Reinforcement Learning from Easy to Hard Tasks Improves LLM Reasoning

ICLR 2026 · Large language models › Reasoning and chain of thought · approach: RL with verifiable rewards for reasoning · 1 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes E2H Reasoner, a curriculum GRPO method that schedules RL post-training tasks from trivial to hard with probabilistic cosine and Gaussian schedulers, with approximate policy iteration convergence and sample-complexity analysis.

Problem

RLVR alone struggles on inherently difficult reasoning tasks because rewards are sparse when base models have low zero-shot accuracy; naive curricula also cause forgetting and overfitting to easy tasks.

Approach

Decompose training data into difficulty levels (trivial, easy, medium, hard) and sample them with a probabilistic schedule (E2H-C cosine, E2H-G Gaussian) that fades easy tasks out over training rather than switching stages abruptly, backed by an approximate policy iteration analysis.

Key results

• On Qwen2.5-1.5B-Instruct, E2H-G reaches 32.9% on Blocksworld Hard versus 21.1% for GRPO (All) and 7.3% OOD versus 2.6%.
• On Countdown with Qwen 1.5B, E2H-G improves Hard accuracy to 41.0% from 18.1% for GRPO (All).
• On LLaMA-3.2-3B, E2H-G gets 44.1% on Blocksworld Hard versus 38.9% for GRPO (All) and 17.6% OOD versus 13.3%.

Surprising findings

• Training only on the hardest examples (GRPO Hard) performs near zero on Blocksworld and underperforms the CoT baseline on MATH for Qwen 1.5B.
• Fixed cosine scheduling (E2H-C) overfits on Blocksworld, collapsing Med/Hard accuracy to near zero, while the Gaussian schedule does not.

Limitations

• E2H-G reports the best of three hyperparameter settings, so reported gains involve selection on the test results.
• Theoretical guarantees rely on strong assumptions (concentrability, bounded curriculum drift, linear function approximation) that do not directly match LLM RL training.
• Experiments are limited to small models (1.5B and 3B) and a handful of tasks.

Experimental setup

Datasets: Blocksworld Countdown MATH GSM8K AQuA

Benchmarks: Blocksworld Countdown MATH AIME24 OlympiadBench

Metrics: accuracy (per difficulty level and OOD split) pass@k

Baselines: GRPO DAPO CoT Self-Evolve traditional curriculum learning

Builds on: GRPO DAPO DeepSeek-R1-Zero Approximate Policy Iteration

Closest papers by meaning

• Reinforcement Learning with Verifiable Rewards Implicitly Incentivizes Correct Reasoning in Base LLMs (ICLR 2026 · similarity 0.92)
• Scheduling Your LLM Reinforcement Learning with Reasoning Trees (ICLR 2026 · similarity 0.91)
• General-Reasoner: Advancing LLM Reasoning Across All Domains (NeurIPS 2025 · similarity 0.91)
• Resource-Efficient Reinforcement for Reasoning Large Language Models via Dynamic One-Shot Policy Refinement (ICML 2026 · similarity 0.91)
• SATURN: SAT-based Reinforcement Learning to Unleash LLMs Reasoning (NeurIPS 2025 · similarity 0.90)
• $\textbf{Re}^{2}$: Unlocking LLM Reasoning via Reinforcement Learning with Re-solving (ICLR 2026 · similarity 0.90)