AI Paper Atlas

Research map / Multi-agent systems and games / LLM agents

HiPER: Hierarchical Plan–Execute RL for Multi-Turn LLM Agents

Jiangweizhi Peng, Yuanxin Liu, Ruida Zhou, Charles Fleming, Zhaoran Wang, Alfredo Garcia, Mingyi Hong

ICML 2026 · Multi-agent systems and games › LLM agents · approach: LLM multi-agent decision making

Read the paperCheck your idea against 137,217 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes HiPER, a hierarchical plan-execute RL method for training multi-turn LLM agents on long-horizon tasks with sparse, delayed rewards.

Problem

Training LLM agents for multi-turn decision-making with sparse and delayed rewards in long-horizon tasks, where flat policies operating at a single time scale struggle.

Approach

Models the LLM agent as a hierarchical policy with a planning level and an execution level operating at different time scales, rather than a flat per-turn action policy.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
switch the base model prompt from ReAct to Plan-Execute without RL training (ablation)Qwen2.5-1.5B-Instruct base · ALFWorldoverall success rate (%): 8.3 -> 2.9Hurt
switch PPO from ReAct to Plan-Execute prompting (ablation)PPO (1.5B) · ALFWorldoverall success rate (%): +13.1%Improved
switch GiGPO from ReAct to Plan-Execute prompting (ablation)GiGPO (1.5B) · ALFWorldoverall success rate (%): +4.4%Improved
add HiPER hierarchical advantage estimation on top of Plan-Execute GiGPO (ablation)GiGPO with Plan-Execute · ALFWorldoverall success rate (%): +4.2%Improved
replace GiGPO with HiPERGiGPO (Qwen2.5-1.5B) · ALFWorldoverall success rate (%): 86.7 -> 95.3Improved
replace GiGPO with HiPERGiGPO (Qwen2.5-1.5B) · WebShopsuccess rate (%): 67.4 -> 71.4Improved
replace GiGPO with HiPERGiGPO (Qwen2.5-7B) · ALFWorldoverall success rate (%): 90.8 -> 97.4Improved
replace GiGPO with HiPERGiGPO (Qwen2.5-7B) · WebShopsuccess rate (%): 75.2 -> 83.3Improved
replace flat PPO/GRPO with HiPER for sample efficiencyPPO/GRPO (Qwen2.5-7B) · ALFWorld validationsteps to reach ~80% success: ~140 steps -> about 50 steps (2.8x)Improved
replace flat PPO/GRPO with HiPER for sample efficiencyPPO/GRPO (Qwen2.5-1.5B) · ALFWorld validationtraining speedup: 2.5x speedupImproved
add RL training (PPO) to the base modelQwen2.5-1.5B-Instruct · ALFWorldoverall success rate (%): 8.3 -> 68.2Improved
add RL training (PPO) to the base modelQwen2.5-1.5B-Instruct · WebShopsuccess rate (%): 5.5 -> 51.5Improved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy