Research map / Reinforcement learning / RL from feedback
TAPO: Dynamic Teacher and Perturbed Answer Injection for Policy Optimization
Maowei Jiang, Zihang Wang, Qi Wang, Peter Búš, Moquan Cheng, Yifan Wang, Quangao Liu, Ruiqi Li, Pengyu Zeng, Ruikai Liu, Alan Liang, Yansong Xu and 3 more
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add Dynamic Teacher Injection (ablation) | GRPO · AIME 2024 | pass@1 accuracy (%): 34.3 -> 39.7 | Improved |
| add Perturbed Answer Injection after DTI (ablation) | GRPO+DTI · AIME 2024 | pass@1 accuracy (%): 39.7 -> 42.8 | Improved |
| add InfoLen reward shaping (ablation) | GRPO+DTI+PAI · AIME 2024 | pass@1 accuracy (%): 42.8 -> 44.3 | Improved |
| full TAPO versus naive GRPO | DeepSeek-R1-Distill-Qwen-1.5B · AIME 2024 | pass@1 accuracy (%): 34.3 -> 44.3 | Improved |
| full TAPO versus base model | DeepSeek-R1-Distill-Qwen-1.5B · AIME 2024 | pass@1 accuracy (%): 28.9 -> 44.3 | Improved |
| full TAPO versus naive GRPO, average | DeepSeek-R1-Distill-Qwen-1.5B · Six math benchmarks average | average accuracy (%): 56.86 -> 62.87 | Improved |
| full TAPO versus naive GRPO | DeepSeek-R1-Distill-Qwen-1.5B · Minerva Math | pass@1 accuracy (%): 28.2 -> 31.6 | Improved |
| InfoLen instead of length-only penalty (ablation) | GRPO-style training · Output length | mean tokens: 3187 -> 2793 | Improved |
| InfoLen versus length-only penalty accuracy (ablation) | GRPO-style training · Math reasoning | accuracy | Improved |
| advantage-level instead of reward-level intervention for injected samples (ablation) | TAPO · Training reward curves | training stability | Improved |
| add DTI versus dynamic sampling (ablation) | GRPO · Training reward | convergence speed | Improved |
| TAPO versus Qwen2.5-7B-SimpleRL | 1.5B model · Minerva Math | pass@1 accuracy (%): 39.7 vs 31.6 | Hurt |
Closest papers by meaning
- Sample More to Think Less: Group Filtered Policy Optimization for Concise Reasoning (ICLR 2026 · similarity 0.89)
- DAPO : Improving Multi-Step Reasoning Abilities of Large Language Models with Direct Advantage-Based Policy Optimization (NeurIPS 2025 · similarity 0.89)
- Act Only When It Pays: Efficient Reinforcement Learning for LLM Reasoning via Selective Rollouts (NeurIPS 2025 · similarity 0.89)
- TraPO: A Semi-Supervised Reinforcement Learning Framework for Boosting LLM Reasoning (ICLR 2026 · similarity 0.88)
- Advancing LLM Reasoning with Natural Language and Numerical Feedback (ICML 2026 · similarity 0.88)
- Accelerating RL for LLM Reasoning with Optimal Advantage Regression (NeurIPS 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
