AI Paper Atlas

Research map / Multi-agent systems and games / LLM agents

WebArbiter: A Generative Reasoning Process Reward Model for Web Agents

Yao Zhang, Shijie Tang, Zeyu Li, Zhen Han, Volker Tresp

ICLR 2026 · Multi-agent systems and games › LLM agents · approach: Agentic benchmarks and orchestration

Read the paper

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes WebArbiter, a generative reasoning process reward model for web agents that judges trajectory steps to support inference-time scaling and address sparse outcome supervision.

Problem

Web agent tasks involve long horizons, multi-step decisions, and irreversible actions; outcome-based supervision is sparse and delayed, often rewarding incorrect trajectories and failing to support inference-time scaling.

Approach

A generative reasoning process reward model for web navigation (WebArbiter) that produces step-level reasoning-based judgments rather than scalar scores from existing WebPRMs; full method details unavailable from the abstract.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
add reasoning distillation before RL (vs cold-start RL) (ablation)WebArbiter-7B · WebPRMBench AverageBoN accuracy (%): 48.33 -> 74.60Improved
add explicit principles to cold-start RL (ablation)Qwen2.5-7B-Instruct + RL · WebPRMBench AverageBoN accuracy (%): 48.33 -> 58.75Improved
use principle-guided distillation instead of distillation without principles (ablation)WebArbiter-7B · WebPRMBench AverageBoN accuracy (%): 55.16 -> 74.60Improved
add RL after reasoning distillation instead of answer-only SFT (ablation)WebArbiter-7B · WebPRMBench AverageBoN accuracy (%): 58.49 -> 74.60Improved
add reasoning distillation versus answer-only SFT, 10K data (ablation)Qwen2.5-7B-Instruct · WebPRMBench AverageBoN accuracy (%): 53.96 -> 60.25Improved
add reasoning distillation versus answer-only SFT, full data (ablation)Qwen2.5-7B-Instruct · WebPRMBench AverageBoN accuracy (%): 55.80 -> 58.49Improved
cold-start RL on Mind2Web-only training data (ablation)Qwen2.5-7B-Instruct · WebArena BoN accuracy (%)BoN accuracy (%): 42.79 -> 35.80Hurt
swap backbone from Qwen2.5 to Qwen3 (4B vs 3B) (ablation)WebArbiter · WebPRMBench AverageBoN accuracy (%): 59.06 -> 72.55Improved
use WebArbiter-7B as PRM in trajectory searchGPT-4o-mini policy · WebArena-Litesuccess rate (%): 23.48 -> 40.52Improved
use WebArbiter-7B instead of WebShepherd-8B in trajectory searchGPT-4o policy · WebArena-Litesuccess rate (%): 40.65 -> 47.01Improved
use GPT-4o-mini as PRM in trajectory searchGPT-4o policy · WebArena-Litesuccess rate (%): 31.90 -> 33.03No clear effect
use WebArbiter vs WebShepherd on CMS (template-driven tasks)GPT-4o-mini policy · WebArena-Lite CMSsuccess rate (%): 45.71 -> 42.86Hurt

Limitations

Experimental setup

Builds on: process reward models

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy