Research map / Reinforcement learning / RL from feedback
RL for Reasoning by Adaptively Revealing Rationales
Mohammad Hossein Amani, Aryo Lotfi, Nicolas Baldwin, Samy Bengio, Mehrdad Farajtabar, Emmanuel Abbe, Robert West
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add adaptive backtracking (AdaBack) on top of SFT+GRPO | Llama-3 1B · GSM8k | test accuracy: 36.7 -> 43.2 | Improved |
| Apply AdaBack directly to the base model instead of GRPO on base | Llama-3 3B · Tensor-2 GSM8k | test accuracy: 0.0 -> 49.2 | Improved |
| Use AdaBack on base model instead of SFT+RL | Llama-3 1B · Base-7 GSM8k | test accuracy: 14.4 -> 18.4 | Improved |
| Add SFT before AdaBack on 3B (ablation) | Llama-3 3B · Tensor-2 GSM8k | test accuracy: 49.2 -> 42.2 | Hurt |
| Add AdaBack on top of SFT+RL at 3B | Llama-3 3B · GSM8k | test accuracy: 72.7 -> 70.7 | Hurt |
| Use AdaBack instead of R3 curriculum (ablation) | Llama-3 base · DeepScaleR | test accuracy: 1B: 6.6 -> 9.0; 3B: 9.6 -> 10.6 | Improved |
| Use AdaBack instead of R3 (ablation) | Llama-3 base · GSM8k | test accuracy: 1B: 41.5 -> 39.2; 3B: 74.2 -> 73.3 | Hurt |
| Use AdaBack instead of R3 (ablation) | Llama-3 base · MATH | test accuracy: 1B: 7.8 -> 9.1; 3B: 19.2 -> 19.1 | Mixed |
| Add AdaBack to RL on instruct-tuned or contaminated models | Llama 3.2 3B-Instruct, Qwen2.5-1.5B · MATH, GSM8k | reward/accuracy | No clear effect |
| Use AdaBack instead of SFT+RL on chain-of-parities | Llama 3.2 1B · synthetic chain-of-parities (L=16, n=1024) | reward: SFT+RL plateau at 0.1; AdaBack learns in under 700 iterations | Improved |
| Vary reward threshold tau (ablation) | AdaBack · GSM8k | training reward and test accuracy | No clear effect |
| Use AdaBack instead of standard RL for pass@k | Llama3-1B · GSM8k pass@k | pass@k | Improved |
Closest papers by meaning
- Training Large Language Models for Reasoning through Reverse Curriculum Reinforcement Learning (ICML 2024 · similarity 0.91)
- Learn to Reason Efficiently with Adaptive Length-based Reward Shaping (ICLR 2026 · similarity 0.89)
- RLAD: Training LLMs to Discover Abstractions for Solving Reasoning Problems (ICLR 2026 · similarity 0.88)
- Curriculum Reinforcement Learning from Easy to Hard Tasks Improves LLM Reasoning (ICLR 2026 · similarity 0.88)
- RuleReasoner: Reinforced Rule-based Reasoning via Domain-aware Dynamic Sampling (ICLR 2026 · similarity 0.88)
- Reshaping Reasoning in LLMs: A Theoretical Analysis of RL Training Dynamics through Pattern Selection (ICLR 2026 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
