Research map / Large language models / Agents, code and math
On Training Large Language Models for Long-Horizon Tasks: An Empirical Study of Horizon Length
Sunghwan Kim, Junhee Cho, Beong-woo Kwak, Taeyoon Kwon, Liang Wang, Nan Yang, Xingxing Zhang, Furu Wei, Jinyoung Yeo
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use macro actions (multiple atomic actions per step) instead of atomic actions | Qwen3-1.7B SFT+RL agent · Sudoku L3-L4 (long goal distance) | average test success rate | Improved |
| use macro actions instead of atomic actions | Qwen3-1.7B SFT+RL agent · Sudoku and Rush Hour L1-L2 | convergence speed and final success rate | Improved |
| restrict macro-action policy to one atomic step per turn (restore long horizon) (ablation) | macro-action policy · Sudoku | training stability | Hurt |
| use flexible-length macro actions rather than fixed-length (n=5) macros (ablation) | GPT-5-mini, Gemini-3-Flash on Sudoku · Sudoku | success rate | Improved |
| add subgoal decomposition with dense subgrid rewards and per-segment returns | RL on Sudoku · Sudoku L3-L4 | average success rate | Improved |
| horizon reduction on WebShop | Qwen3-1.7B RL agent · WebShop | training stability and average success rate | Improved |
| scale model from 1.7B to 4B with atomic actions (ablation) | RL on Sudoku L3-L4 · Sudoku L3-L4 | training stability | No clear effect |
| add horizon reduction to a 4B model | Qwen3 4B RL on Sudoku · Sudoku L3-L4 | collapse and performance | Improved |
| use GRPO-style optimizer instead of REINFORCE variant, with and without horizon reduction (ablation) | RL on Sudoku · Sudoku | success rate over training | Improved |
| train on L1-L4 range with macro actions and test on longer unseen goal distances | macro-action policy · Sudoku and Rush Hour unseen horizons | success rate | Improved |
| add curriculum: train on short distances (4-9) then continue on long (10-12) (ablation) | RL on Rush Hour · Rush Hour | success rate | Improved |
Closest papers by meaning
- Scaling Long-Horizon Agent via Context Folding (ICML 2026 · similarity 0.91)
- The Illusion of Diminishing Returns: Measuring Long Horizon Execution in LLMs (ICLR 2026 · similarity 0.91)
- Can large language models explore in-context? (NeurIPS 2024 · similarity 0.90)
- Language Control Diffusion: Efficiently Scaling through Space, Time, and Tasks (ICLR 2024 · similarity 0.90)
- LARFT: Closing the Cognition-Action Gap for Length Instruction Following in Large Language Models (ICML 2026 · similarity 0.89)
- Large Language Models Think Too Fast To Explore Effectively (NeurIPS 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
