Research map / Reinforcement learning / RL from feedback
Experience Augmented Policy Optimization for LLM Reasoning
Jinda Lu, Kexin Huang, Junkang Wu, Shuo Yang, Jinghan Li, Chiyu Ma, Shaohang Wei, Xiang Wang, Guoyin Wang, Jingren Zhou
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add smoothed response-level importance sampling to EAPO (ablation) | EAPO · Math average (Qwen-2.5-Math-7B) | Pass@1 (%): 42.76 -> 43.34 | Improved |
| add smoothed importance sampling to EAPO (ablation) | EAPO · GPQA (Qwen-2.5-Math-7B) | accuracy (%): 41.98 -> 43.18 | Improved |
| add smoothed importance sampling to EAPO (ablation) | EAPO · Math average (Qwen-2.5-Math-7B) | Pass@16 (%): 62.37 -> 62.41 | No clear effect |
| use token-level experience instead of trajectory-level experience (positive samples only) (ablation) | EAPO · AIME'24 | Mean@32 | Improved |
| include negative experience-augmented samples in addition to positive-only filtering (ablation) | EAPO · AIME'24 | Mean@32 | Hurt |
| raise resampling threshold tau to 0.8 or above (ablation) | EAPO · AIME'24 | Mean@32 | Hurt |
| vary resampling threshold tau within 0.3 to 0.5 (ablation) | EAPO · AIME'24 | Mean@32: 0.381 at tau=0.5 | No clear effect |
| use EAPO experience injection instead of plain DAPO | DAPO · Math average (Qwen-3-8B-Base) | Pass@1 (%): 45.31 -> 49.71 | Improved |
| use EAPO instead of DAPO | DAPO · Math average (Qwen-2.5-Math-7B) | Pass@1 (%): 40.56 -> 43.34 | Improved |
| use EAPO instead of DAPO | DAPO · GPQA and MMLU-Pro (Qwen-2.5-Math-7B) | average accuracy (%): 41.68 -> 45.38 | Improved |
| replace token-level resampling with trajectory replay of successful trajectories | EAPO · Math average (Qwen-2.5-Math-7B) | Pass@1 (%): 43.34 -> 42.08 | Hurt |
| add on-policy distillation from RL-optimized policy (OPD) instead of EAPO | DAPO · Math average (Qwen-2.5-Math-7B) | Pass@1 (%): 40.56 -> 40.64 | No clear effect |
Closest papers by meaning
- Resource-Efficient Reinforcement for Reasoning Large Language Models via Dynamic One-Shot Policy Refinement (ICML 2026 · similarity 0.91)
- Accelerating RL for LLM Reasoning with Optimal Advantage Regression (NeurIPS 2025 · similarity 0.91)
- One-Way Policy Optimization for Self-Evolving LLMs (ICML 2026 · similarity 0.90)
- Reshaping Reasoning in LLMs: A Theoretical Analysis of RL Training Dynamics through Pattern Selection (ICLR 2026 · similarity 0.90)
- Reinforcement Learning with Verifiable Rewards Implicitly Incentivizes Correct Reasoning in Base LLMs (ICLR 2026 · similarity 0.90)
- What are Key Factors for Updates in RL for LLM Reasoning? (NeurIPS 2026 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
