Research map / Multi-agent systems and games / LLM agents
GAIA: A Data Flywheel System for Training GUI Test-Time Scaling Critic Models
Shaokang Wang, Pei Fu, Ruoceng Zhang, Shaojie Zhang, Xiuwen Xi, Jiahui Yang, Bin Qin, Ying Huang, Zhenbo Luo, Jian Luan
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add ICM critic with best-of-8 selection | UI-TARS 1.0 · AndroidControl-High | SR: 58.2 -> 67.5 | Improved |
| add ICM critic with best-of-8 selection | UI-TARS 1.5 · GUI-Odyssey | SR: 32.9 -> 47.8 | Improved |
| add ICM-r2 (second flywheel round) instead of ICM (ablation) | UI-TARS 1.5 · GUI-Odyssey | SR: 47.8 -> 50.2 | Improved |
| add ICM-r2 instead of ICM (ablation) | UI-TARS 1.5 · AndroidControl-High | SR: 64.5 -> 65.6 | Improved |
| add ICM-r2 instead of ICM (ablation) | UI-TARS 1.0 · AndroidControl-High | SR: 67.5 -> 67.1 | No clear effect |
| add ICM to closed-source GPT-4o | GPT-4o · AndroidControl-High | SR: 13.2 -> 17.0 | Improved |
| add ICM to UI-TARS 1.5 grounding | UI-TARS 1.5 · ScreenSpotV2 | average accuracy: 90.8 -> 90.2 | No clear effect |
| add ICM to Qwen2.5-VL grounding | Qwen2.5-VL-7B · ScreenSpotV2 | average accuracy: 65.0 -> 70.4 | Improved |
| use intuitive binary critic instead of reasoning critic (ablation) | GAIA critic · GUI-Odyssey with UI-TARS 1.5 | SR: 44.1 -> 47.8 | Improved |
| use intuitive binary critic instead of reasoning critic (ablation) | GAIA critic · GAIA critic test set | critic accuracy: 70.82 -> 83.19 | Improved |
| train critic on real-agent action data versus UI-Genie-RM (ablation) | Qwen2.5-VL-7B · AndroidControl-High | delta SR: 0.3 -> 2.8 | Improved |
| add second flywheel round data (ablation) | critic accuracy · GAIA critic test set | critic accuracy: 83.19 -> 83.56 | Improved |
Closest papers by meaning
- OS-Oracle: A Comprehensive Framework for Cross-Platform GUI Critic Models (CVPR 2026 · similarity 0.92)
- GTA1: GUI Test-time Scaling Agent (ICLR 2026 · similarity 0.89)
- ShowUI: One Vision-Language-Action Model for GUI Visual Agent (CVPR 2025 · similarity 0.88)
- GUI-G1: Understanding R1-Zero-Like Training for Visual Grounding in GUI Agents (NeurIPS 2025 · similarity 0.88)
- GUI-Shift: Enhancing VLM-Based GUI Agents through Self-supervised Reinforcement Learning (ICLR 2026 · similarity 0.88)
- SE-GUI: Enhancing Visual Grounding for GUI Agents via Self-Evolutionary Reinforcement Learning (NeurIPS 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
