Research map / Large language models / Reasoning and chain of thought

AdaCuRL: Adaptive Curriculum Reinforcement Learning with Invalid Sample Mitigation and Historical Revisiting

AAAI 2026 · Large language models › Reasoning and chain of thought · approach: RL with verifiable rewards for reasoning · 1 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes AdaCuRL, a curriculum GRPO framework with coarse-to-fine difficulty estimation, adaptive bucket scheduling with historical revisiting, and sparse/reset-reference KL, plus a self-paced re-estimation variant (Re-AdaCuRL), improving Qwen2.5-VL and Qwen2.5-Math reasoning.

Problem

GRPO training on mixed-difficulty data suffers from Gradient Starvation (all-0/all-1 rewards zero the advantage) and Policy Degradation (KL penalty dominates on invalid samples), and existing curriculum methods suffer from difficulty mismatch, manual schedules, and forgetting.

Approach

Estimates difficulty from the policy's own pass rates in a coarse-to-fine manner, schedules buckets adaptively via a competence score driven by accuracy rewards while merging (revisiting) earlier buckets, computes the KL term only for groups with non-zero advantage, and resets the reference model after each bucket update; a self-paced round re-estimates difficulty with the updated policy.

Key results

• On Qwen2.5-VL-3B, AdaCuRL improves the math-reasoning average from 40.64 (GRPO) to 43.81, and Re-AdaCuRL reaches 45.18.
• On Qwen2.5-Math-7B, AdaCuRL raises the average over five math benchmarks from 38.63 (GRPO) to 44.16.
• Removing sparse KL, reference reset, or revisiting each lowers the Qwen2.5-VL-3B math average (42.42, 41.58, 41.74 vs 43.81).

Surprising findings

• Plain SFT on the open-source data degrades Qwen2.5-VL-7B math average (47.48 to 43.54), and GRPO gives only small gains on the multimodal models.
• Training only on easy or only on hard samples both underperform the mixed curriculum.

Limitations

• Coarse difficulty estimation is only moderately accurate (72%, 53%, 64% per group).
• Hyperparameters such as bucket count and generation settings were manually tuned per model.
• Re-AdaCuRL uses a fixed threshold (0.2) for discarding easy samples without study.

Experimental setup

Datasets: CLEVR CLEVR-Math Geo3K GeoMverse GeoQA+ IconQA Super-CLEVR TabMWP UniGeo GEOS WeMath SceMQA

Benchmarks: DynaMath MathVista MathVision MathVerse LogicVista MMStar MMMU HallusionBench AI2D MMVET AIME24 AMC23

Metrics: accuracy average accuracy Wilcoxon signed-rank p-value

Baselines: GRPO SFT naive CL

Builds on: GRPO Self-Paced Learning Open-RS R1-V

Closest papers by meaning

• TTRL: Test-Time Reinforcement Learning (NeurIPS 2025 · similarity 0.89)
• CurES: From Gradient Analysis to Efficient Curriculum Learning for Reasoning LLMs (ICLR 2026 · similarity 0.89)
• Rethinking the Sampling Criteria in Reinforcement Learning for LLM Reasoning: A Competence-Difficulty Alignment Perspective (AAAI 2026 · similarity 0.88)
• ProRL: Prolonged Reinforcement Learning Expands Reasoning Boundaries in Large Language Models (NeurIPS 2025 · similarity 0.88)
• On Predictability of Reinforcement Learning Dynamics for Large Language Models (ICLR 2026 · similarity 0.88)
• Improving Data Efficiency for LLM Reinforcement Fine-tuning Through Difficulty-targeted Online Data Selection and Rollout Replay (NeurIPS 2025 · similarity 0.88)