Research map / Large language models / Reasoning and chain of thought

A Task-centric Theory for Iterative Self-Improvement with Easy-to-Hard Curricula

ICML 2026 · Large language models › Reasoning and chain of thought · approach: Test-time compute scaling · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Gives finite-sample, task-centric theory for reward-filtered iterative self-improvement (MLE on accepted samples), showing feasibility and improvement conditions under which easy-to-hard curricula beat fixed mixtures, validated on a synthetic shortest-path task and GSM8K/DeepMind Mathematics.

Problem

Finite-sample theory for multi-step iterative self-improvement of LLMs on reward-verified self-generated data, and when easy-to-hard task scheduling provably beats fixed mixtures.

Approach

Models each self-improvement round as MLE on a reward-filtered distribution, deriving finite-sample expected-reward lower bounds via Hellinger/TV arguments and an iterated map F; extends to easy-to-hard schedules with power-law difficulty separation, giving explicit feasibility and improvement regions in initialization, difficulty ratio, and budget.

Key results

• On GSM8K, Qwen3-8B Pass@1 rises from 27.52% to 30.67% over three self-improvement iterations, while Llama-3.2-1B-Instruct declines (-0.28).
• On DeepMind Mathematics, Qwen3-8B improves from 85.11% to 92.78%, while Qwen3-32B drops by 1.20 points from an initial 97.89%.
• Easy-to-hard curricula outperform the fixed-mixture baseline only for moderate adjacent difficulty ratios; gains peak at interior values of the mixing parameter rho.

Surprising findings

• Self-improvement can deteriorate for overly easy tasks or small models, e.g., Llama-3.2-1B and 3B lose accuracy on GSM8K across iterations.
• Qwen3-32B slightly degrades on DeepMind Mathematics after the first iteration despite high initial accuracy.

Limitations

• Bounds depend on a finite model class assumption and a reward-acceptance coupling assumption that may be hard to verify.
• Theory uses a single-candidate or first-accepted sampling scheme and a local neighborhood of initialization.

Experimental setup

Datasets: GSM8K DeepMind Mathematics synthetic shortest path graphs

Benchmarks: GSM8K DeepMind Mathematics synthetic shortest path

Metrics: Pass@1

Baselines: fixed-mixture training

Builds on: rejection sampling fine-tuning self-consuming loop theory

Closest papers by meaning

• B-STaR: Monitoring and Balancing Exploration and Exploitation in Self-Taught Reasoners (ICLR 2025 · similarity 0.90)
• Progress or Regress? Self-Improvement Reversal in Post-training (ICLR 2025 · similarity 0.90)
• Self-Improvement in Language Models: The Sharpening Mechanism (ICLR 2025 · similarity 0.90)
• Mind the Gap: Examining the Self-Improvement Capabilities of Large Language Models (ICLR 2025 · similarity 0.90)
• Toward Self-Improvement of LLMs via Imagination, Searching, and Criticizing (NeurIPS 2024 · similarity 0.89)
• Rethinking Reward Models for Multi-Domain Test-Time Scaling (NeurIPS 2026 · similarity 0.88)