Research map / Large language models / Reasoning and chain of thought

CurES: From Gradient Analysis to Efficient Curriculum Learning for Reasoning LLMs

ICLR 2026 · Large language models › Reasoning and chain of thought · approach: Efficient and adaptive thinking · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes CurES, a gradient-analysis-derived curriculum for RLVR that reallocates prompt sampling and rollout budgets using Beta-posterior accuracy estimates, improving GRPO/RPP efficiency on math reasoning with Qwen2.5-Math 1.5B and 7B.

Problem

Existing RLVR curriculum and data-selection methods ignore prompt difficulty variation or waste compute on uniform sampling and online filtering, lowering training efficiency for LLM reasoning.

Approach

Derives from a natural-gradient/Cramér-Rao analysis that prompt sampling should follow a softmax over sqrt(p(1-p)) and rollout counts should be proportional to the per-prompt gradient standard deviation; estimates per-prompt accuracy online with a Beta-Binomial posterior.

Key results

• CurES-GRPO averages 44.94 vs GRPO 41.64 on Qwen2.5-Math-1.5B (+3.30), and 52.41 vs 47.59 on 7B (+4.82).
• CurES-GRPO reaches GRPO's peak performance in 5.5x fewer steps; CurES-RPP reaches RPP peak 1.75x faster.
• CurES exceeds the best prior sample-efficient method by +2.12 points on average across eight math benchmarks.

Surprising findings

• Increasing the parameter estimation sample size N' or rollout budget n does not yield proportional performance gains, so small values suffice.

Limitations

• Theoretical analysis relies on approximations (Taylor expansion, trust region, trace-based variance minimization) rather than exact guarantees.
• Experiments limited to Qwen2.5-Math models and math reasoning benchmarks.
• Iterative dataset partitioning resets estimates each iteration, a heuristic to handle distribution shift.

Experimental setup

Datasets: Numina-Math

Benchmarks: MATH500 GSM8K Gaokao-EN 2023 Minerva OlympiadBench AIME 2024 AIME 2025 AMC 2023

Metrics: Pass@1 Average@16 convergence speed (steps to peak)

Baselines: GRPO REINFORCE++ Speed-RL GVM

Builds on: GRPO REINFORCE++ GVM Speed-RL VERL

Closest papers by meaning

• h1: Bootstrapping LLMs to Reason over Longer Horizons via Reinforcement Learning (ICML 2026 · similarity 0.91)
• Rectifying LLM Thought from Lens of Optimization (ICLR 2026 · similarity 0.90)
• Curriculum Reinforcement Learning from Easy to Hard Tasks Improves LLM Reasoning (ICLR 2026 · similarity 0.90)
• QuestA: Expanding Reasoning Capacity in LLMs via Question Augmentation (ICLR 2026 · similarity 0.90)
• Accelerating RL for LLM Reasoning with Optimal Advantage Regression (NeurIPS 2025 · similarity 0.90)
• General-Reasoner: Advancing LLM Reasoning Across All Domains (NeurIPS 2025 · similarity 0.89)