Research map / Reinforcement learning / Imitation learning
GUI-SAGE: Enhancing GUI Automation with Self-Explanatory Learning
Fei Tang, Zhangxuan Gu, Zhengxi Lu, Shangzhan Zhang, Zhengwen Zeng, Shuheng Shen, Changhua Meng, Yuchen Yan, Wenqi Zhang, Yongliang Shen, Weiming Lu, Yueting Zhuang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add self-explanation samples conditioned on ground-truth action to GRPO rollouts (ablation) | Vanilla-GRPO (Qwen2.5-VL-3B) · AndroidControl-Low + High | Avg SR (%): 80.1 -> 83.1 | Improved |
| add expert demonstrations from Qwen2.5-VL-72B instead of self-explanations (ablation) | Vanilla-GRPO (Qwen2.5-VL-3B) · AndroidControl-Low + High | Avg SR (%): 80.1 -> 81.2 | Improved |
| replace expert demonstrations with self-explanations (ablation) | GUI-SAGE-3B · AndroidControl-Low + High | Avg SR (%): 81.2 -> 83.1 | Improved |
| add Entropy-Modulated Credit Assignment (EMCA) to self-explanation (ablation) | GUI-SAGE-3B · AndroidControl-Low + High | Avg SR (%): 83.1 -> 84.2 | Improved |
| add EMCA to vanilla GRPO (ablation) | Vanilla-GRPO · AndroidControl-Low + High | Avg SR (%): 80.1 -> 80.6 | Improved |
| add EMCA to expert demonstration training (ablation) | Expert-CoT GRPO · AndroidControl-Low + High | Avg SR (%): 81.2 -> 81.9 | Improved |
| add explicit reasoning before action prediction (ablation) | Vanilla-GRPO · AndroidControl-Low + High | Avg SR (%): 78.5 -> 80.1 | Improved |
| add partial hint of action type only (ATH) (ablation) | Vanilla-GRPO · AndroidControl-Low + High | Avg SR (%): 80.1 -> 81.4 | Improved |
| add partial hint of action parameters only (APH) (ablation) | Vanilla-GRPO · AndroidControl-Low + High | Avg SR (%): 80.1 -> 81.8 | Improved |
| use complete ground-truth action hint instead of partial hints (ablation) | hint-based GRPO · AndroidControl-Low + High | Avg SR (%): 81.8 -> 83.1 | Improved |
| set reward weights to equal w1=w2=1.0 instead of 0.8/0.2 or 0.2/0.8 (ablation) | GUI-SAGE-3B · AndroidControl + GUI-Odyssey | Avg SR (%): 83.5 / 83.7 -> 84.2 | No clear effect |
| measure zero-advantage fraction in vanilla GRPO early training | Vanilla-GRPO · AndroidControl/GUI-Odyssey training, steps 0-100 | zero-advantage samples (%): 73.2 | Hurt |
Closest papers by meaning
- GUI-Shift: Enhancing VLM-Based GUI Agents through Self-supervised Reinforcement Learning (ICLR 2026 · similarity 0.90)
- Learn where to Click from Yourself: On-Policy Self-Distillation for GUI Grounding (NeurIPS 2026 · similarity 0.87)
- Training High-Level Schedulers with Execution-Feedback Reinforcement Learning for Long-Horizon GUI Automation (CVPR 2026 · similarity 0.87)
- UI-Genie: A Self-Improving Approach for Iteratively Boosting MLLM-based Mobile GUI Agents (NeurIPS 2025 · similarity 0.87)
- History-Aware Reasoning for GUI Agents (AAAI 2026 · similarity 0.87)
- SE-GUI: Enhancing Visual Grounding for GUI Agents via Self-Evolutionary Reinforcement Learning (NeurIPS 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
