Research map / Multi-agent systems and games / LLM agents

SEAgent: Self-Evolving Computer Use Agent with Autonomous Learning from Experience

ICML 2026 · Multi-agent systems and games › LLM agents · approach: Agentic benchmarks and orchestration

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes SEAgent, a self-evolving computer-use agent framework that explores unfamiliar OSWorld software, uses a fine-tuned World State Model for step-level rewards, a guidebook-based Curriculum Generator, and GRPO plus adversarial imitation, followed by specialist-to-generalist distillation.

Problem

Computer use agents struggle with novel or specialized software that lacks human-annotated demonstrations, and existing reward models for CUA trajectories have poor judgment precision and low reward density.

Approach

Combine a trajectory-level World State Model that labels each action as correct or failure, a self-maintained software guidebook that drives curriculum task generation, and a policy update mixing GRPO on successful actions with adversarial imitation on failure actions; then distill specialists into a generalist via SFT and further RL.

Key results

• SEAgent specialist-to-generalist raises UI-TARS-7B-DPO average OSWorld success rate on five software apps from 11.3% to 34.5%.
• Specialist-to-generalist (34.5%) outperforms specialist ensemble RL (32.2%) and general RL (30.6%).
• World State Model (Qwen2.5-VL-7B fine-tuned) reaches 71.6% precision on AgentRewardBench with entire-process screenshots, versus 25.4% for base Qwen2.5-VL-7B and 72.1% for GPT-4o.

Surprising findings

• Feeding full-process screenshots to base Qwen2.5-VL models sharply degrades judgment precision (e.g., 64.1% to 25.4% for 7B), though the fine-tuned World State Model recovers it.
• Directly training a generalist across all software underperforms specialists, motivating the specialist-to-generalist strategy.

Limitations

• Self-evolution is bounded by the reward quality of the World State Model rather than real environment signals.
• Evaluated tasks are relatively simple, requiring under about 20 steps for a human expert.
• Gains on UI-TARS-1.5 for OSWorld are moderate because that model was already trained on OSWorld-like data.

Experimental setup

Datasets: OSWorld Chrome trajectories AgentRewardBench self-generated curriculum tasks on five OSWorld software apps

Benchmarks: OSWorld AgentRewardBench ScienceBoard

Metrics: success rate precision negative predictive value average precision

Baselines: GPT-4o GPT-4V Gemini-Pro-1.5 Claude3.7 Sonnet Gemini-Pro-2.5 UI-TARS-7B-DPO UI-TARS-72B-DPO DigiRL WebRL NNetNav

Builds on: GRPO DPO DigiRL WebRL NNetNav

Closest papers by meaning

• Agent S: An Open Agentic Framework that Uses Computers Like a Human (ICLR 2025 · similarity 0.88)
• MemEvolve: Meta-Evolution of Agent Memory Systems (ICML 2026 · similarity 0.88)
• OpenCUA: Open Foundations for Computer-Use Agents (NeurIPS 2025 · similarity 0.87)
• OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environments (NeurIPS 2024 · similarity 0.87)
• Darwinian Memory: A Training-Free Self-Regulating Memory System for GUI Agent Evolution (ICML 2026 · similarity 0.87)
• Autonomous Continual Learning for Environment Adaptation of Computer-Use Agents (NeurIPS 2026 · similarity 0.87)