Research map / Multi-agent systems and games / LLM agents
GUI Exploration Lab: Enhancing Screen Navigation in Agents via Multi-Turn Reinforcement Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces GE-Lab, a configurable GUI simulation engine with full environment access, and uses it to show that SFT, then single-turn RL, then multi-turn RL progressively improve screen navigation of Qwen2.5-VL-7B agents on ID, OOD, and interactive benchmarks.
Problem
Real-world GUI platforms are complex and proprietary, so they lack the full screen layout and navigation-graph information needed to train and systematically benchmark screen navigation agents.
Approach
A simulation engine that procedurally generates tree-structured screen navigation graphs with randomized, uncommon icons and provides exact reward signals, enabling a comparison of SFT, single-turn RL (GRPO with a composite reward), and multi-turn RL with sparse goal-reaching reward.
Key results
- MT-RL achieves Pass@1 17.47 and Pass@5 25.16 on the interactive benchmark, versus 14.30 / 20.86 for SFT.
- ST-RL improves OOD overall accuracy to 63.06 from 55.45 for SFT (a 13.7% relative gain); MT-RL reaches 63.25.
- After continual training on real-world data, MT-RL-Continue-Train averages 72.03 across real-world grounding and interactive benchmarks, versus 70.87 for SFT-Continue-Train.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use single-turn reinforcement learning instead of SFT | Qwen2.5-VL-7B-Instruct · Env-Base OOD | Overall: 55.45 vs. 63.06 | Helped |
| Use single-turn reinforcement learning instead of SFT | Qwen2.5-VL-7B-Instruct · Interactive benchmark | Pass@1: 14.30 and 17.22 | Helped |
| Use multi-turn reinforcement learning instead of SFT | Qwen2.5-VL-7B-Instruct · Interactive benchmark | Pass@5: 20.86 and 25.16 | Helped |
| Use multi-turn reinforcement learning instead of SFT | Qwen2.5-VL-7B-Instruct · In-distribution benchmark | Overall: 98.89 -> 67.33 | Hurt |
| Use multi-turn reinforcement learning instead of ST-RL | Qwen2.5-VL-7B-Instruct · Env-Base OOD | Overall: 63.06 -> 63.25 | Helped |
| Use multi-turn reinforcement learning instead of ST-RL | Qwen2.5-VL-7B-Instruct · Interactive benchmark | Pass@5: 22.34 -> 25.16 | Helped |
| Replace the icons while preserving labels and positions (Env-Image) | MT-RL · OOD environment | Overall: 63.25 -> 34.17 | Hurt |
| Alter icon labels while preserving appearance and layout (Env-Name) | MT-RL · OOD environment | Overall: 63.25 -> 32.74 | Hurt |
| Randomize icon positions (Env-Position) | MT-RL · OOD environment | Overall: 63.25 -> 42.79 | Hurt |
| Add task-irrelevant noise icons (Env-Noise) | MT-RL · OOD environment | Overall: 63.25 -> 44.56 | Hurt |
| Initialize ST-RL from an early SFT stage rather than a later SFT stage (ablation) | ST-RL · ST-RL training | cumulative reward | Helped |
| Add the format matching reward (ablation) | ST-RL · GE-Lab navigation | navigation success rate | Helped |
Surprising findings
- MT-RL scores lower than SFT on in-distribution static tests (overall 67.33 vs 98.89), which the authors attribute to avoiding overfitting.
- RL initialized from early SFT checkpoints (1 epoch) reaches higher final reward than from later SFT checkpoints, suggesting more SFT is not always better.
Limitations
- Simulated environment with simplified icons, limited action space (click, complete for training), and no text-heavy or noisy real-world UI complexity.
- Real-world generalization evaluated only on a static benchmark with a restricted set of actions and continual training on 24k real samples.
- Results from a single base model (Qwen2.5-VL-7B) and a single environment configuration.
Experimental setup
Datasets: Env-Base synthetic GE-Lab navigation data AITW AITZ AMEX Mind2Web ScreenSpot ScreenSpot-v2 FuncPred MoTIF Refexp LlamaTouch VWB AG
Benchmarks: GE-Lab Env-Base ID static test GE-Lab OOD environments (Env-Image, Env-Name, Env-Position, Env-Noise) GE-Lab interactive benchmark (2,162 tasks) real-world static benchmark (1,569 samples) ScreenSpot ScreenSpot-v2 FuncPred MoTIF Refexp VWB AG VWB EG AndroidWorld
Metrics: accuracy edge accuracy path accuracy Pass@1 Pass@5 success rate
Baselines: GPT-4o-2024-11-20 Claude-3.7-Sonnet Gemini-2.0-flash-thinking SFT ST-RL UI-TARS UI-R1
Builds on: GRPO DeepSeek-R1 ReAct RAGEN VAGEN UI-R1
Closest papers by meaning
- SE-GUI: Enhancing Visual Grounding for GUI Agents via Self-Evolutionary Reinforcement Learning (NeurIPS 2025 · similarity 0.92)
- GUI-Eyes: Tool-Augmented Perception for Visual Grounding in GUI Agents (AAAI 2026 · similarity 0.90)
- GUI-G1: Understanding R1-Zero-Like Training for Visual Grounding in GUI Agents (NeurIPS 2025 · similarity 0.90)
- GUI-Shift: Enhancing VLM-Based GUI Agents through Self-supervised Reinforcement Learning (ICLR 2026 · similarity 0.90)
- GuirlVG: Incentivize GUI Visual Grounding via Empirical Exploration on Reinforcement Learning (ICLR 2026 · similarity 0.89)
- Scaling, Benchmarking, and Reasoning of Vision-Language Agents for Mobile GUI Navigation (ICML 2026 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.