Research map / Reinforcement learning / Visual and representation RL
Milestone-Guided Policy Learning for Long-Horizon Language Agents
Zixuan Wang, Yuchen Yan, Hongxing Li, Teng Pan, Dingming Li, Ruiqing Zhang, Weiming Lu, Jun Xiao, Yueting Zhuang, Yongliang Shen
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use BEACON milestone-anchored credit instead of GRPO on long ALFWorld tasks | Qwen2.5-1.5B-Instruct · ALFWorld (Long tasks) | success rate (%): 53.5 -> 92.9 | Improved |
| use BEACON instead of GiGPO | Qwen2.5-1.5B-Instruct · ScienceWorld | success rate (%): 25.8 -> 45.3 | Improved |
| add temporal reward shaping and milestone credit to improve sample utilization | GRPO · ALFWorld | effective sample utilization (%): 23.7 -> 82.0 | Improved |
| add temporal decay (gamma=0.95) instead of gamma=0 (ablation) | BEACON · ALFWorld | success rate (%): 81.2 -> 91.4 | Improved |
| add temporal decay (gamma=0.95) instead of gamma=0 (ablation) | BEACON · WebShop | success rate (%): 62.1 -> 75.6 | Improved |
| use uniform shaping (gamma=1) instead of decayed shaping (ablation) | BEACON · ALFWorld | success rate (%): 81.2 (gamma=0) -> 71.8 (gamma=1) | Hurt |
| use detected milestones instead of random partition (ablation) | BEACON · ALFWorld | success rate (%): 74.2 -> 91.4 | Improved |
| drop 50% of milestones (ablation) | BEACON · ALFWorld | success rate (%): 91.4 -> 82.8 | Hurt |
| add segment-level advantage to GRPO's trajectory-level advantage (ablation) | BEACON · ALFWorld | success rate (%): 72.8 -> 91.4 | Improved |
| add trajectory-level advantage to segment-level advantage (ablation) | BEACON · ALFWorld | success rate (%): 23.4 -> 91.4 | Improved |
| add trajectory-level advantage to segment-level advantage (ablation) | BEACON · WebShop | success rate (%): 67.9 -> 75.6 | Improved |
| use BEACON (gamma=0.95) instead of SFT on oracle trajectories | Qwen2.5-1.5B-Instruct · ALFWorld | success rate (%): 43 -> 91.4 | Improved |
Closest papers by meaning
- Latent Reward: LLM-Empowered Credit Assignment in Episodic Reinforcement Learning (AAAI 2025 · similarity 0.88)
- A Subgoal-driven RL Framework for Improving Long-Horizon Web Agents (NeurIPS 2026 · similarity 0.88)
- Learning Agentic Policy from Action Guidance (NeurIPS 2026 · similarity 0.88)
- SRPO: Self-Reflective Policy Optimization for Long-Horizon Reasoning (ICML 2026 · similarity 0.88)
- Prioritize the Process, Not Just the Outcome: Rewarding Latent Thought Trajectories Improves Reasoning in Looped Language Models (ICML 2026 · similarity 0.88)
- StraTA: Incentivizing Agentic Reinforcement Learning with Strategic Trajectory Abstraction (NeurIPS 2026 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
