Research map / Multi-agent systems and games / LLM agents

ProRe: A Proactive Reward System for GUI Agents via Reasoner–Actor Collaboration

ICLR 2026 · Multi-agent systems and games › LLM agents · approach: Self-evolving language agent learning

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes ProRe, a proactive reward system in which a general-purpose reasoner (Gemini-2.5-Pro) schedules state-probing tasks executed by domain-specific evaluator agents (V-Droid) that interact with the GUI, then judges success via chain-of-claims reasoning.

Problem

Rewarding GUI agents is hard: rule-based verifiers need costly hand-written checks, and trajectory-based LLM-as-a-judge has low accuracy due to incomplete state observability and lack of GUI domain knowledge.

Approach

Replace passive trajectory judging with proactive state probing: a general reasoner plans probing tasks, domain-specific evaluator agents actively navigate the environment to collect key states, and the reasoner compares policy and evaluator claims (confirming, contradicting, complementary) to output a binary reward; iterative and parallel probing add test-time scaling.

Key results

• ProRe reaches 93.7% average reward accuracy over 3K+ trajectories, up to 5.3% higher accuracy and 19.4% higher F1 than the best baseline.
• On PC/web pilot experiments, ProRe reaches 92.0% on OSWorld and 93.5% on OSWorld-Chrome reward accuracy.
• Guided by ProRe, M3A (GPT-4o) test-time scaling success rate increases by 22.4%, and V-Droid's SR improves from 56.5% to 67.2%.

Surprising findings

• Fine-tuned small GUI agents (V-Droid-8B) outperform large generalist agentic systems as evaluators, while Qwen3-VL-4B yields notably lower accuracy.
• Parallel state probing yields larger gains than iterative probing, as the general reasoner struggles to exploit intermediate observations for refining probes.

Limitations

• Per-task cost is roughly 4-6x higher than baselines (about $0.063 vs $0.010-0.017).
• Accuracy depends on evaluator capability; small evaluators like Qwen3-VL-4B generalize poorly to unseen probing tasks.
• Iterative probing gains are limited because the general reasoner lacks GUI knowledge to refine probes from intermediate observations.

Experimental setup

Datasets: AndroidWorld AndroidLab MobileAgentBench OSWorld

Benchmarks: AndroidWorld AndroidLab MobileAgentBench OSWorld OSWorld-Chrome

Metrics: reward accuracy F1 success rate

Baselines: DigiRL DistRL WebRL Step-Critic M3A

Builds on: LLM-as-a-judge V-Droid

Closest papers by meaning

• Pushing Forward Pareto Frontiers of Proactive Agents with Behavioral Agentic Optimization (ICML 2026 · similarity 0.87)
• Proactive Agent: Shifting LLM Agents from Reactive Responses to Active Assistance (ICLR 2025 · similarity 0.87)
• UniClawBench: A Universal Benchmark for Proactive Agents on Real-World Tasks (NeurIPS 2026 · similarity 0.87)
• Rewarding Progress: Scaling Automated Process Verifiers for LLM Reasoning (ICLR 2025 · similarity 0.86)
• ProAct: A Benchmark and Multimodal Framework for Structure-Aware Proactive Response (ICML 2026 · similarity 0.86)
• PROACT-Agent: Progressive Runtime Oversight and Active Circuit-breaking for Real-Time Safety (NeurIPS 2026 · similarity 0.86)