Research map / Multi-agent systems and games / LLM agents
LongHorizonUI: A Unified Framework for Robust long-horizon Task Automation of GUI Agent
Bin Kang, Shaoguo Wen, Yifei Bi, Shunlong Wu, Xinbin Yuan, Rui Shao, Junle Wang, Zhuotao Tian
Read the paperCheck your idea against 137,217 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes LongHorizonUI, a unified framework to improve the robustness and sustained reliability of MLLM-based GUI agents on long-horizon tasks in dynamic environments.
Problem
MLLM-based GUI agents perform well on short-term GUI tasks but lack robustness on complex long-horizon tasks in dynamic environments.
Approach
A unified framework intended to overcome core limitations of existing GUI agents for sustained reliability over long-horizon tasks; specific mechanism not stated in the abstract provided.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use LongHorizonUI instead of UI-TARS-1.5 | GUI agent · LongGUIBench General-High | SR: 21.8 -> 52.3 | Improved |
| use LongHorizonUI instead of UI-TARS-1.5 | GUI agent · LongGUIBench Game-Low | SR: 69.5 -> 83.9 | Improved |
| use LongHorizonUI instead of InfiGUI-R1 | GUI agent · ScreenSpot | avg accuracy: 87.5 -> 90.4 | Improved |
| use LongHorizonUI instead of UI-TARS-7B | GUI agent · ScreenSpot-V2 | avg accuracy: 84.7 -> 86.2 | Improved |
| use LongHorizonUI instead of GUI-R1-7B | GUI agent · AndroidControl and GUI-Odyssey | avg TM/SR: 63.2 -> 65.5 | Improved |
| use LongHorizonUI instead of UI-TARS-72B | GUI agent · OSWorld 50 steps | success rate: 24.6 -> 29.4 | Improved |
| add the icon detector to the perceiver (ablation) | Multimodal Enhanced Perceiver · LongGUIBench | step-completion rate: +6.1% | Improved |
| add OCR to the perceiver (ablation) | Multimodal Enhanced Perceiver · LongGUIBench | step-completion rate: +2.3% | Improved |
| add relative-coordinate fallback to index execution (ablation) | Compensating Action Executor · LongGUIBench | task completion: 81.4 -> +1.2% | Improved |
| add absolute-coordinate fallback to index execution (ablation) | Compensating Action Executor · LongGUIBench | task completion: 81.4 -> +2.5% | Improved |
| add historical-coordinate fallback to index execution (ablation) | Compensating Action Executor · LongGUIBench | task completion: 81.4 -> +3.9% | Improved |
| raise IoU merge threshold from 0.6 to 0.8 (ablation) | Perceiver · LongGUIBench | task success rate: 82.7 -> 83.9 | Improved |
Closest papers by meaning
- MMBench-GUI: A Unified Hierarchical Evaluation Framework for Multi-Platform GUI Agents (CVPR 2026 · similarity 0.90)
- Training High-Level Schedulers with Execution-Feedback Reinforcement Learning for Long-Horizon GUI Automation (CVPR 2026 · similarity 0.90)
- History-Aware Reasoning for GUI Agents (AAAI 2026 · similarity 0.90)
- HATS: Hardness-Aware Trajectory Synthesis for GUI Agents (CVPR 2026 · similarity 0.89)
- GUI-CEval: A Hierarchical and Comprehensive Chinese Benchmark for Mobile GUI Agents (CVPR 2026 · similarity 0.89)
- See, Think, Act: Teaching Multimodal Agents to Effectively Interact with GUI by Identifying Toggles (CVPR 2026 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
