Research map / Multi-agent systems and games / LLM agents
From Local Skills to Long-Horizon Tasks: Progressive Skill Exploration for LLM Web Agents
Xueyang Feng, Xiaohe Bo, Xu Chen, Quanyu Dai, Feng Wen
Read the paperCheck your idea against 137,217 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a progressive skill exploration approach for training LLM web agents, moving from local skills to long-horizon tasks to address sparse rewards and rare successful trajectories.
Problem
Training LLM web agents with naive exploration is hard because successful trajectories are rare and sparse terminal rewards give little useful feedback from failed trajectories.
Approach
Draws on skill acquisition theory to train agents progressively, first exploring local skills and then composing them into long-horizon web tasks, so that feedback is available beyond terminal rewards.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| train with SCOPE progressive exploration instead of WebRL | Llama3.1-8B · WebArena-Lite | total success rate: 45.8 -> 54.3 | Improved |
| train with SCOPE instead of RFT | Qwen2.5-3B · WebArena-Lite | total success rate: 36.5 -> 41.6 | Improved |
| train with SCOPE instead of RFT | Llama3.1-8B · WebVoyager | total success rate: 37.3 -> 42.8 | Improved |
| train with SCOPE instead of GRPO | Llama3.1-8B · WebShop | success rate: 76.2 -> 85.2 | Improved |
| train with SCOPE instead of GRPO | Qwen2.5-3B · WebShop | success rate: 80.8 -> 84.6 | Improved |
| use teacher intervention (OPTI) vs self intervention (OPSI) (ablation) | SCOPE · WebArena-Lite, Qwen2.5-3B | total success rate: OPTI 41.6 vs OPSI 40.2 | Mixed |
| use self intervention (OPSI) vs teacher intervention (OPTI) (ablation) | SCOPE · WebArena-Lite, Llama3.1-8B | total success rate: OPSI 54.3 vs OPTI 53.6 | Mixed |
| add more outer training rounds (1 to 3) (ablation) | SCOPE · WebVoyager | success rate: total gain of 4.25% | Improved |
| add more inner exploration iterations (ablation) | SCOPE · WebVoyager | success rate: 33.40% -> 43.41% | Improved |
| use self-managed memory model instead of external DeepSeek-V3.2 (ablation) | SCOPE · WebArena/WebVoyager | final improvement | No clear effect |
| remove auxiliary information c (ablation) | SCOPE (OPTI) · WebArena-Lite, Llama3.1-8B | total success rate: 53.6 -> 52.7 | No clear effect |
| remove skill discovery, orchestration or expansion (ablation) | SCOPE · component ablation | success rate | Improved |
Closest papers by meaning
- A Subgoal-driven RL Framework for Improving Long-Horizon Web Agents (NeurIPS 2026 · similarity 0.91)
- WebRL: Training LLM Web Agents via Self-Evolving Online Curriculum Reinforcement Learning (ICLR 2025 · similarity 0.90)
- WebFactory: Automated Compression of Foundational Language Intelligence into Grounded Web Agents (ICLR 2026 · similarity 0.90)
- SkillMaster: Toward Autonomous Skill Mastery in LLM Agents (NeurIPS 2026 · similarity 0.90)
- WebGym: Scaling Training Environments for Long-Horizon Visual Web Agents with Realistic Tasks (CVPR 2026 · similarity 0.90)
- SkillOS: Learning Skill Curation for Self-Evolving Agents (NeurIPS 2026 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
