Research map / Multi-agent systems and games / LLM agents

TreeCUA: Efficiently Scaling GUI Automation with Tree-Structured Verifiable Evolution

ICML 2026 · Multi-agent systems and games › LLM agents · approach: GUI agent memory and trajectories · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes TreeCUA, a multi-agent, tree-structured verifiable exploration pipeline that synthesizes GUI trajectories (50K filtered trajectories, 708K step samples) and trains Qwen2.5-VL-7B with two-stage SFT plus a tree-derived DPO stage (TreeCUA-DPO), reaching 34.6% and 36.6% on OSWorld-Verified.

Problem

Scaling GUI planning data for computer-use agents is costly because human-annotated trajectories are scarce and exploration of different apps repeats the same early steps, causing redundancy and low diversity.

Approach

Organize exploration of apps as trees with node replay (reset and re-execute action history) to reuse shared prefixes, use world-knowledge guidance, global prefix memory and step-level verification to get diverse, verified trajectories, and derive DPO preference pairs from sibling branches at no extra annotation cost.

Key results

• TreeCUA-7B reaches 34.6% overall success on OSWorld-Verified, above OpenCUA-7B (24.3%) and UltraCUA-7B (28.9%) at 50 steps.
• TreeCUA-DPO-7B reaches 36.6% on OSWorld-Verified, with gains concentrated in TB (+20.0) and Code (+13.1) but drops in Impress (-10.6).
• On a self-constructed 120-task OOD benchmark, TreeCUA-DPO-7B scores 30.8% vs 0.8% for Qwen2.5-VL-7B.

Surprising findings

• DPO improved some domains but regressed on Impress (-10.6 points) and OS and Calc, which the authors attribute to continuous visual precision demands.
• Scaling with the same Qwen2.5-VL-7B backbone, OpenCUA and ScaleCUA data gave only 20.2% on OSWorld versus 34.6% for TreeCUA data in the ablation.

Limitations

• Exploration agent relies on Claude-4.5-Sonnet and auxiliary agents on GPT-4o-mini, so data generation depends on closed-source APIs.
• DPO gains are uneven across applications, regressing on visually precise apps like Impress.
• OOD benchmark is self-constructed and judged by GPT-4o with manual verification, limiting comparability.

Experimental setup

Datasets: TreeCUA synthetic trajectories human expert task examples AndroidControl

Benchmarks: OSWorld-Verified OOD benchmark (120 tasks, 6 apps) AndroidControl

Metrics: success rate ROSCOE reasoning scores Type-Token Ratio Jaccard similarity

Baselines: Qwen2.5-VL-7B UI-TARS-1.5-7B OpenCUA-7B ScaleCUA-7B UltraCUA-7B Seed-1.8 Claude-Sonnet-4.5

Builds on: DPO OpenCUA ScaleCUA ROSCOE

Closest papers by meaning

• TongUI: Internet-Scale Trajectories from Multimodal Web Tutorials for Generalized GUI Agents (AAAI 2026 · similarity 0.89)
• MMBench-GUI: A Unified Hierarchical Evaluation Framework for Multi-Platform GUI Agents (CVPR 2026 · similarity 0.88)
• ProBench: Benchmarking GUI Agents with Accurate Process Information (AAAI 2026 · similarity 0.88)
• GUI-Xplore: Empowering Generalizable GUI Agents with One Exploration (CVPR 2025 · similarity 0.88)
• HATS: Hardness-Aware Trajectory Synthesis for GUI Agents (CVPR 2026 · similarity 0.87)
• GTA1: GUI Test-time Scaling Agent (ICLR 2026 · similarity 0.87)