Research map / Multi-agent systems and games / LLM agents
Learning from Failure: Inference-Time Self-Improvement for Computer-Use Agents
Xueqiao Sun, Yuhui Zhang, Xiaohan Wang, Ludwig Schmidt, Serena Yeung-Levy
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add inference-time failure-derived patches (full method) | OpenCUA-72B · OSWorld (100 steps) | success rate (%): 42.3 -> 48.9 | Improved |
| add visual search post-action verification (ablation) | OpenCUA-72B · OSWorld small set | success rate (%): 41.67 -> 47.22 | Improved |
| add terminal execution (ablation) | OpenCUA-72B · OSWorld small set | success rate (%): 41.67 -> 47.19 | Improved |
| add knowledge support (manual retriever and search) (ablation) | OpenCUA-72B · OSWorld small set | success rate (%): 41.67 -> 44.44 | Improved |
| add repetition detection and recovery mode (ablation) | OpenCUA-72B · OSWorld small set | success rate (%): 41.67 -> 44.40 | Improved |
| combine all four strategies (ablation) | OpenCUA-72B · OSWorld small set | success rate (%): 41.67 -> 52.74 | Improved |
| apply same loop to OpenCUA-32B | OpenCUA-32B · OSWorld (100 steps) | success rate (%): 34.5 -> 38.2 | Improved |
| apply same loop to GUI-Owl-32B | GUI-Owl-32B · OSWorld (100 steps) | success rate (%): 19.0 -> 21.3 | Improved |
| transfer OSWorld-derived patches to AndroidControl | Qwen3-VL-32B-Instruct · AndroidControl | score: 28.37 -> 36.23 | Improved |
| transfer OSWorld-derived patches to WebVoyager | Qwen3-VL-32B-Instruct · WebVoyager | success rate: 23.80 -> 27.90 | Improved |
| transfer OSWorld-derived patches to ScreenSpotPro | Qwen3-VL-32B-Instruct · ScreenSpotPro | accuracy: 27.50 -> 30.74 | Improved |
Closest papers by meaning
- AgenTracer: Who Is Inducing Failure in the LLM Agentic Systems? (ICLR 2026 · similarity 0.89)
- AgentForesight: Online Auditing for Early Failure Prediction in Multi-Agent Systems (NeurIPS 2026 · similarity 0.89)
- Beyond LLM-Based Reasoning: Lightweight GNNs for Agent Failure Attribution (NeurIPS 2026 · similarity 0.88)
- CORRECT: COndensed eRror RECognition via knowledge Transfer in multi-agent systems (ICML 2026 · similarity 0.88)
- TACT: Mitigating Overthinking and Overacting in Coding Agents via Activation Steering (NeurIPS 2026 · similarity 0.88)
- Meta$^n$: Recursive Self-Improvement through Emergent Depth (NeurIPS 2026 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
