Research map / Multi-agent systems and games / LLM agents

WebChain: A Large-Scale Human-Annotated Dataset of Real-World Web Interaction Traces

CVPR 2026 · Multi-agent systems and games › LLM agents · approach: Agentic benchmarks and orchestration · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces WebChain, a 31,725-trajectory human-annotated real-world web interaction dataset with triple alignment (visual, AX/DOM, action), and a Dual Mid-Training recipe (spatial grounding RL + CoT-SFT before long-horizon RL) evaluated on WebChainBench and public GUI benchmarks.

Problem

Web and GUI agents lack large-scale, reproducible, human-annotated trajectory data from real live websites, since synthetic collection fails on authentication, CAPTCHAs and anti-bot defenses, and proprietary datasets are not shareable.

Approach

A large human-annotated live-web trajectory dataset with aligned screenshots, accessibility/DOM structure and precise action grounding, plus a two-stage mid-training recipe that separates spatial grounding RL and CoT-SFT from long-horizon RL.

Key results

• On WCB-L, Qwen2.5-VL-3B with Directly LCRL reaches 0.603 while adding CoT-SFT and SGRL (Both) reaches 0.658, versus 0.487 for GUI-R1-3B.
• WebChain-LCRL-7B +SGRL+CoT-SFT reports 81.4 overall score across GUI benchmarks in the table, versus 74.2 for GUI-R1-7B.
• Performance on WCB-L improves with WebChain data scale (4k, 20k, full subsets) for Qwen2.5-VL-3B LCRL.

Surprising findings

• Reasoner prompting for spatial grounding lowered downstream long-horizon RL performance compared with non-RP initialization.
• Approximately 50% of the dataset... (not applicable; none stated)

Limitations

• Experiments are reported largely on Qwen2.5-VL-3B with limited detail on RL algorithm and hyperparameters.
• Inconsistent dataset size reporting (150k steps in scaling study vs 318k total steps).

Experimental setup

Datasets: WebChain AndroidControl GUI-Act-Web GUI-Odyssey OmniAct

Benchmarks: WebChainBench AndroidControl-High AndroidControl-Low GUI-Act-Web GUI-Odyssey OmniAct-Desktop OmniAct-Web

Metrics: action type accuracy grounding rate step success rate success rate on WCB-L

Baselines: Qwen2.5-VL-3B Qwen2.5-VL-7B GUI-R1-3B GUI-R1-7B

Builds on: GUI-R1 Qwen2.5-VL Mind2Web WebArena

Closest papers by meaning

• TimeWarp: Evaluating Web Agents by Revisiting the Past (NeurIPS 2026 · similarity 0.87)
• WebWorld: A Large-Scale World Model for Web Agent Training (ICML 2026 · similarity 0.87)
• Video2GUI: Synthesizing Large-Scale Interaction Trajectories for Generalized GUI Agent Pretraining (ICML 2026 · similarity 0.86)
• Watch and Learn: Learning to Use Computers from Online Videos (CVPR 2026 · similarity 0.86)
• WebGym: Scaling Training Environments for Long-Horizon Visual Web Agents with Realistic Tasks (CVPR 2026 · similarity 0.86)
• WARC-Bench: Web Archive based Benchmark for GUI Subtask Executions (ICLR 2026 · similarity 0.86)