Research map / Reinforcement learning / Visual and representation RL
ComputerRL: Scaling End-to-End Online Reinforcement Learning for Computer Use Agents
Hanyu Lai, Xiao Liu, Yanxiao Zhao, Han Xu, Hanchen Zhang, Bohao Jing, Yanyu Ren, Shuntian Yao, Yuxiao Dong, Jie Tang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use API-GUI action space instead of GUI only (ablation) | GPT-4o agent · OSWorld | average success: 11.2 -> 26.2 | Improved |
| add behavior cloning cold start (ablation) | Qwen2.5-14B · OSWorld | average success: 15.2 -> 31.9 | Improved |
| add RL phase 1 (ablation) | Qwen2.5-14B · OSWorld | average success: 31.9 -> 42.0 | Improved |
| add Entropulse (SFT on successful rollouts) after RL1 (ablation) | Qwen2.5-14B · OSWorld | average success: 42.0 -> 41.5 | No clear effect |
| add RL phase 2 after Entropulse (ablation) | Qwen2.5-14B · OSWorld | average success: 41.5 -> 45.8 | Improved |
| add Entropulse before second RL phase vs plain continued RL (ablation) | ComputerRL · training reward and entropy | reward | Improved |
| raise clipping threshold (DAPO-style) to slow entropy decline (ablation) | ComputerRL RL · entropy / learning | policy improvement | Hurt |
| use RL-trained ComputerRL 9B vs strongest proprietary baseline | GLM-4.1V-9B-Thinking · OSWorld | success rate: 42.9 -> 48.9 | Improved |
| train with ComputerRL on GLM-4.1V vs GLM-4-9B backbone | AutoGLM-OS · OSWorld | success rate: 48.1 -> 48.9 | No clear effect |
| use ComputerRL agent vs OpenAI o3 on PowerPoint tasks | GLM-4.1V-9B-Thinking · OfficeWorld PPT | success: 41.7 -> 41.7 | No clear effect |
Closest papers by meaning
- WebRL: Training LLM Web Agents via Self-Evolving Online Curriculum Reinforcement Learning (ICLR 2025 · similarity 0.88)
- SRL: Scaling Distributed Reinforcement Learning to Over Ten Thousand Cores (ICLR 2024 · similarity 0.87)
- DistRL: An Asynchronous Distributed Reinforcement Learning Framework for On-Device Control Agent (ICLR 2025 · similarity 0.87)
- AstraFlow: Dataflow-Oriented Reinforcement Learning for Agentic LLMs (NeurIPS 2026 · similarity 0.87)
- SE-GUI: Enhancing Visual Grounding for GUI Agents via Self-Evolutionary Reinforcement Learning (NeurIPS 2025 · similarity 0.87)
- AutoTool: Automatic Scaling of Tool-Use Capabilities in RL via Decoupled Entropy Constraints (ICLR 2026 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
