Research map / Multi-agent systems and games / LLM agents
WebDS: An End-to-End Benchmark for Web-based Data Science
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces WebDS, an 870-task end-to-end web-based data science benchmark across 29 websites, evaluating web agents (BrowserUse, AgentOccam, GPT/Claude/Qwen models) and humans with success rate and LLM-judge scores.
Problem
Existing web agent benchmarks focus on simple browsing tasks without data manipulation, while data science benchmarks use static structured data and ignore web navigation, so end-to-end web-based data science workflows are not evaluated.
Approach
A hand-written benchmark of 870 multi-hop, multi-website, multimodal data science tasks requiring browsing, data acquisition, tool-based analysis, and optional web actions, with a live track and a dockerized reproducible track, plus fine-grained LLM-judge scoring over full trajectories.
Key results
- BrowserUse with GPT-5.1 achieves the highest success rate on WebDS-live at 22.2%, while BrowserUse with GPT-4o reaches 12.9% and Qwen2.5-72B 13.2%, versus 81% on WebVoyager.
- AgentOccam drops from 45.7% on WebArena to 4.8% on WebDS-live.
- Human participants achieve about 90% success rate under the same environment and tool constraints.
Surprising findings
- BrowserUse with Qwen2.5-72B outperforms the same setup with GPT-4o, suggesting raw model capability is not the main bottleneck.
- Larger model capacity does not necessarily yield higher performance; GPT-4o performs similarly to GPT-4o-mini and Qwen2.5-72B on the base framework.
Limitations
- Live-web track is subject to website changes, affecting reproducibility.
- Agents show high failure rates on multi-hop and multi-site tasks, with failure modes such as groundedness errors and repetition.
- Action-based tasks are underrepresented in the current task suite.
Experimental setup
Datasets: WebDS tasks (870 hand-written tasks across 29 websites)
Benchmarks: WebDS-live WebDS-dockerized WebVoyager WebArena
Metrics: success rate LLM-judge score (1-5) status agreement score standard deviation
Baselines: AgentOccam BrowserUse (Qwen2.5-72B) BrowserUse (GPT-4o) BrowserUse (GPT-5.1) GPT-4o GPT-4o-mini Claude Sonnet-3.5 Claude Haiku-3.5 Qwen2.5-72B-Instruct Qwen2.5-VL-72B-Instruct
Builds on: WebArena WebVoyager BrowserUse AgentOccam BrowserGym
Closest papers by meaning
- AutoData: A Multi-Agent System for Open Web Data Collection (NeurIPS 2025 · similarity 0.86)
- The FineWeb Datasets: Decanting the Web for the Finest Text Data at Scale (NeurIPS 2024 · similarity 0.85)
- DSBench: How Far Are Data Science Agents from Becoming Data Science Experts? (ICLR 2025 · similarity 0.84)
- WebChain: A Large-Scale Human-Annotated Dataset of Real-World Web Interaction Traces (CVPR 2026 · similarity 0.84)
- WebWeaver: Structuring Web-Scale Evidence with Dynamic Outlines for Open-Ended Deep Research (ICLR 2026 · similarity 0.84)
- WebWorld: A Large-Scale World Model for Web Agent Training (ICML 2026 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.