Research map / Reinforcement learning / RL from feedback
Replay Failures as Successes: Sample-Efficient Reinforcement Learning for Instruction Following
Kongcheng Zhang, QI YAO, Shunyu Liu, Wenjian Zhang, Min Cen, Yang Zhou, Wenkai Fang, Yiru Zhao, Baisheng Lai, Mingli Song
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add adaptive select-then-rewrite replay selection instead of random replay (ablation) | HiR Llama-3.2-3B-Instruct · IFEval | instruction-level accuracy: 79.9 -> 83.6 | Improved |
| add adaptive select-then-rewrite replay selection instead of random replay (ablation) | HiR Qwen2.5-7B-Instruct · IFEval | instruction-level accuracy: 79.5 -> 81.0 | Improved |
| add adaptive select-then-rewrite replay selection instead of random replay (ablation) | HiR Qwen3-4B-Instruct-2507 · IFEval | instruction-level accuracy: 85.2 -> 86.3 | Improved |
| add adaptive replay selection instead of random replay (ablation) | HiR Qwen2.5-7B-Instruct · FollowBench | instruction-level accuracy: 66.2 -> 65.1 | Hurt |
| add adaptive replay selection instead of random replay (ablation) | HiR Llama-3.2-3B-Instruct · IFBench | instruction-level accuracy: 28.2 -> 30.4 | Improved |
| add hindsight instruction replay to RL with constraint-level reward | RL-CR Llama-3.2-3B-Instruct · IFEval | instruction-level accuracy: 79.1 -> 83.6 | Improved |
| add hindsight instruction replay to RL with constraint-level reward | RL-CR Qwen3-4B-Instruct-2507 · IFBench | instruction-level accuracy: 36.9 -> 40.5 | Improved |
| train with DPO instead of base model | Qwen2.5-7B-Instruct · IFEval | instruction-level accuracy: 72.6 -> 66.9 | Hurt |
| train with SFT instead of base model | Qwen2.5-7B-Instruct · CFBench | instruction-level accuracy: 57.5 -> 53.1 | Hurt |
| train HiR on instruction following data | Qwen3-4B-Instruct-2507 · MMLU-Pro (out-of-domain) | accuracy: 69.6 -> 67.2 | Hurt |
| train HiR on instruction following data | Qwen3-4B-Instruct-2507 · MATH-500 (out-of-domain) | accuracy: 86.8 -> 88.2 | No clear effect |
| train HiR on instruction following data | Qwen2.5-7B-Instruct · MATH-500 (out-of-domain) | accuracy: 76.6 -> 76.6 | No clear effect |
Closest papers by meaning
- IFDECORATOR: Wrapping Instruction Following Reinforcement Learning with Verifiable Rewards (NeurIPS 2026 · similarity 0.90)
- Checklists Are Better Than Reward Models For Aligning Language Models (NeurIPS 2025 · similarity 0.90)
- Generalizing Verifiable Instruction Following (NeurIPS 2025 · similarity 0.88)
- A Theoretical Framework for Partially-Observed Reward States in RLHF (ICLR 2025 · similarity 0.88)
- Incentivizing Reasoning for Advanced Instruction-Following of Large Language Models (NeurIPS 2025 · similarity 0.88)
- BLEUBERI: BLEU is a surprisingly effective reward for instruction following (NeurIPS 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
