Research map / Multi-agent systems and games / LLM agents
Iris: Breaking GUI Complexity with Adaptive Focus and Self-Refining
Zhiqi Ge, Juncheng Li, Xinglei Pang, Minghe Gao, Kaihang Pan, Wang Lin, Hao Fei, Wenqiao Zhang, Siliang Tang, Yueting Zhuang
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add Self-Refining Dual Learning (SRDL) (ablation) | Iris · ScreenSpot (avg) | accuracy: 64.7 -> 74.6 | Improved |
| add visual hard case mining to SRDL (ablation) | Iris · ScreenSpot (avg) | accuracy: 71.4 -> 74.6 | Improved |
| add functional hard case mining to SRDL (ablation) | Iris · ScreenSpot (avg) | accuracy: 72.1 -> 74.6 | Improved |
| add Information-Sensitive Cropping (ISC) (ablation) | Iris · ScreenSpot (avg) | efficiency: 300% efficiency improvement | Improved |
| add SRDL (ablation) | Iris · ScreenSpot (avg) | accuracy: 10% accuracy gain | Improved |
| use ISC instead of down-resize or AnyRes (ablation) | Iris · ScreenSpot visual complexity | accuracy and tokens | Improved |
| Iris vs SeeClick with ISC and SRDL | Iris · ScreenSpot (avg) | accuracy: 53.4 -> 74.6 | Improved |
| Iris vs SeeClick | Iris · GroundUI-1K (total) | accuracy: 61.1 -> 71.3 | Improved |
| Iris vs SeeClick | Iris · Mind2Web Cross-Task | Step SR: 25.5 -> 32.0 | Improved |
| Iris vs SeeClick | Iris · AITW (overall) | score: 59.3 -> 63.6 | Improved |
| Iris vs UGround on desktop icon/widget grounding | Iris · ScreenSpot Desktop Icon/Widget | accuracy: 63.6 -> 57.5 | Hurt |
| Iris vs MindAct (uses HTML) on Mind2Web | Iris · Mind2Web Cross-Task | Step SR: 52.0 -> 32.0 | Hurt |
Closest papers by meaning
- Navigating the Digital World as Humans Do: Universal Visual Grounding for GUI Agents (ICLR 2025 · similarity 0.88)
- ShowUI: One Vision-Language-Action Model for GUI Visual Agent (CVPR 2025 · similarity 0.87)
- UI-Vision: A Desktop-centric GUI Benchmark for Visual Perception and Interaction (ICML 2025 · similarity 0.86)
- iSHIFT: Lightweight Slow-Fast GUI Agent with Adaptive Perception (CVPR 2026 · similarity 0.86)
- Towards GUI Agents: Vision-Language Diffusion Models for GUI Grounding (CVPR 2026 · similarity 0.86)
- SE-GUI: Enhancing Visual Grounding for GUI Agents via Self-Evolutionary Reinforcement Learning (NeurIPS 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
