Research map / Reinforcement learning / RL from feedback
Hierarchy-of-Groups Policy Optimization for Long-Horizon Agentic Tasks
Shuo He, Lang Feng, Qi Wei, Xin Cheng, Lei Feng, Bo An
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add hierarchical groups (G1:K) beyond step-level group (ablation) | HGPO K=2 (Qwen2.5-1.5B) · ALFWorld in-distribution | success rate (%): 90.88 -> 92.77 | Improved |
| add hierarchical groups (G1:K) beyond step-level group (ablation) | HGPO K=4 (Qwen2.5-1.5B) · ALFWorld in-distribution | success rate (%): 91.14 -> 94.85 | Improved |
| add hierarchical groups to WebShop (ablation) | HGPO K=2 (Qwen2.5-1.5B) · WebShop | task success rate (%): 73.24 -> 71.54 | Hurt |
| use adaptive weights (alpha=1) instead of uniform weights (ablation) | HGPO K=4 (Qwen2.5-1.5B) · ALFWorld in-distribution | success rate (%): 92.12 -> 94.85 | Improved |
| use adaptive weights instead of uniform weights (ablation) | HGPO K=2 (Qwen2.5-1.5B) · ALFWorld in-distribution | success rate (%): 94.33 -> 92.77 | Hurt |
| use alpha=2 instead of alpha=1 (ablation) | HGPO K=4 (Qwen2.5-1.5B) · ALFWorld in-distribution | success rate (%): 94.85 -> 93.55 | Hurt |
| add trajectory-level advantage to final estimate (ablation) | HGPO K=4 (Qwen2.5-1.5B) · ALFWorld in-distribution | success rate (%): 94.85 -> 91.27 | Hurt |
| HGPO vs GiGPO | HGPO K=4 (Qwen2.5-1.5B) · WebShop | task success rate (%): 73.24 -> 78.12 | Improved |
| HGPO vs GiGPO on larger model | HGPO K=4 (Qwen2.5-7B) · ALFWorld in-distribution | success rate (%): 95.63 -> 95.96 | No clear effect |
| increase memory length K from 2 to 4 | HGPO (Qwen2.5-1.5B) · ALFWorld in-distribution | success rate (%): 92.77 -> 94.85 | Improved |
| use only Oracle steps for policy optimization (ablation) | Stepwise GRPO · ALFWorld/WebShop | success rate | Hurt |
Closest papers by meaning
- Group-in-Group Policy Optimization for LLM Agent Training (NeurIPS 2025 · similarity 0.93)
- Group-Graph Policy Optimization for Long-Horizon Agentic Reinforcement Learning (NeurIPS 2026 · similarity 0.93)
- Stronger-MAS: Multi-Agent Reinforcement Learning for Collaborative LLMs (ICLR 2026 · similarity 0.91)
- Listwise Policy Optimization: Group-based RLVR as Target-Projection on the LLM Response Simplex (NeurIPS 2026 · similarity 0.90)
- Agentic Reinforced Policy Optimization (ICLR 2026 · similarity 0.88)
- Solving the Granularity Mismatch: Hierarchical Preference Learning for Long-Horizon LLM Agents (ICLR 2026 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
