Research map / Reinforcement learning / RL from feedback
On the Emergence of Implicit Curriculum in RLVR Learning Dynamics
Yu Huang, Zixin Wen, Yuejie Chi, Yuting Wei, Aarti Singh, Yingbin Liang, Yuxin Chen
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| train with outcome-only RL on a short horizon (L = 5) (ablation) | one-layer attention with frozen pretrained MLP, REINFORCE · synthetic Z96 state tracking, fixed length | reward and attention concentration | Improved |
| train with outcome-only RL on longer fixed horizons (L = 15, 45) (ablation) | one-layer attention with frozen pretrained MLP, REINFORCE · synthetic Z96 state tracking, fixed length | reward and attention concentration | Hurt |
| mix horizons with moderate difficulty ratio R = 3 (L in {5,15,45}) (ablation) | mixed-difficulty REINFORCE · synthetic Z96 state tracking, mixed length | reward trajectory across horizons | Improved |
| mix horizons with large difficulty ratio R = 7 (L in {5,35}) (ablation) | mixed-difficulty REINFORCE · synthetic Z96 state tracking, mixed length | reward trajectory across horizons | Hurt |
| replace outcome-based RL with supervised fine-tuning with step-level supervision (theory) | one-layer transformer · long-horizon composition beyond critical horizon | learnability | Improved |
Closest papers by meaning
- Reshaping Reasoning in LLMs: A Theoretical Analysis of RL Training Dynamics through Pattern Selection (ICLR 2026 · similarity 0.91)
- The Reasoning Boundary Paradox: How Reinforcement Learning Constrains Language Models (NeurIPS 2026 · similarity 0.90)
- Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model? (NeurIPS 2025 · similarity 0.90)
- Detecting RLVR Training Data via Structural Convergence of Reasoning (NeurIPS 2026 · similarity 0.89)
- Probing RLVR Training Instability through the Lens of Objective-Level Hacking (ICML 2026 · similarity 0.89)
- Reinforcement Learning with Verifiable Rewards Implicitly Incentivizes Correct Reasoning in Base LLMs (ICLR 2026 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
