Research map / Multi-agent systems and games / Human-AI and goal-conditioned
Efficient Agent Training for Computer Use
Yanheng He, Jiahe Jin, Pengfei Liu
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| augment human trajectories with 9 Claude-synthesized alternative actions per step (Trajectory Boost) instead of human data only (ablation) | PC Agent-E (Qwen2.5-VL-72B) · WindowsAgentArena-V2 | success rate (%): 17.2 -> 36.0 | Improved |
| use Trajectory Boost instead of direct end-to-end distillation from Claude 3.7 Sonnet (ablation) | PC Agent-E · WindowsAgentArena-V2 | success rate (%): 26.2 -> 36.0 | Improved |
| add 312 human trajectories to direct distillation data (ablation) | Direct distillation · WindowsAgentArena-V2 | success rate (%): 26.2 -> 26.9 | No clear effect |
| offline single-step synthesis rather than online trajectory collection for data generation | Data synthesis · collecting 3120 trajectories | collection time: 900 hours -> 3 hours | Improved |
| fine-tune Qwen2.5-VL-72B with PC Agent-E data | Qwen2.5-VL-72B · WindowsAgentArena-V2 | success rate (%): 14.9 -> 36.0 | Improved |
| fine-tune the smaller Qwen2.5-VL-7B with the same data | Qwen2.5-VL-7B · WindowsAgentArena-V2 | success rate (%): 5.0 -> 6.4 | Improved |
| raise step limit from 15 to 30 (ablation) | PC Agent-E · WindowsAgentArena-V2 | success rate (%): 26.9 -> 36.0 | Improved |
| raise step limit from 30 to 50 (ablation) | PC Agent-E · WindowsAgentArena-V2 | success rate (%): 36.0 -> 31.4 | Hurt |
| train on Windows data only, evaluate on Linux OSWorld | PC Agent-E · OSWorld feasible tasks | success rate (%): 4.4 -> 10.9 | Improved |
| train PC Agent-E on infeasible tasks outcomes | PC Agent-E · OSWorld infeasible tasks | success rate (%): 86.7 -> 63.3 | Hurt |
Closest papers by meaning
- AgentTrek: Agent Trajectory Synthesis via Guiding Replay with Web Tutorials (ICLR 2025 · similarity 0.88)
- Watch and Learn: Learning to Use Computers from Online Videos (CVPR 2026 · similarity 0.88)
- AgentSynth: Scalable Task Generation for Generalist Computer-Use Agents (ICLR 2026 · similarity 0.87)
- AgentBrew: Offline Tool-Use Agent Learning from Raw Real-World Trajectories (NeurIPS 2026 · similarity 0.87)
- Agent S: An Open Agentic Framework that Uses Computers Like a Human (ICLR 2025 · similarity 0.87)
- Learn-by-interact: A Data-Centric Framework For Self-Adaptive Agents in Realistic Environments (ICLR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
