Research map / Multi-agent systems and games / LLM agents
AgentWebBench: Benchmarking Multi-Agent Coordination in Agentic Web
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces AgentWebBench, a benchmark where a user agent coordinates with 100 website-specific content agents (18.4M ClueWeb22-B documents) across web search, recommendation, QA, and deep research, comparing three coordination strategies over seven LLMs.
Problem
Agentic web access is shifting from centralized retrieval to decentralized coordination between user agents and provider-controlled content agents, but no benchmark evaluates multi-agent coordination for common web information needs.
Approach
Model the Agentic Web as a decentralized ecosystem where each website has its own content agent with scoped retrieval; compare embedding-based selection, LLM-prompted selection with tool retrieval, and full agent-to-agent multi-agent coordination against a centralized retrieval baseline.
Key results
- Multi-Agent with Gemini-3 reaches 67.92% accuracy on QA versus 60.38% for Classical, while on web search Classical with Gemini-3 gets N@3 52.21 vs 44.34 for Multi-Agent.
- Thinking mode raises DeepSeek-V3.2 Multi-Agent web search N@3 from 12.26% (non-thinking) to 33.42% (thinking).
- Under Multi-Agent coordination, citations concentrate on a small set of domains (e.g., wikipedia.org, sciencedirect.com) and Qwen3-4B has 31.12% no-reference responses in deep research.
Surprising findings
- Multi-agent coordination can outperform the centralized Classical baseline on question answering with strong models (Gemini-3 67.92% vs 60.38% accuracy).
- Web recommendation is near-zero recall for most methods across all models and strategies.
Limitations
- Small QA (53 queries) and web recommendation evaluation sets limit statistical power.
- Corpus limited to 100 websites from ClueWeb22-B English subset rather than the open web.
- Failure attribution uses deterministic rules and an LLM adjudicator (GPT-5-nano) for QA.
Experimental setup
Datasets: ClueWeb22-B English subset MS MARCO Web Search test set ORBIT DeepResearchGym
Benchmarks: AgentWebBench web search AgentWebBench web recommendation AgentWebBench question answering AgentWebBench deep research
Metrics: NDCG@k Recall@k accuracy token-level F1 KPR KPC Clarity Insight
Baselines: Classical Classic IR Tool-Based Embedding Selection Tool-Based Prompted Selection
Builds on: MS MARCO Web Search DeepResearchGym ClueWeb22-B ORBIT GEO
Closest papers by meaning
- REAL: Benchmarking Autonomous Agents on Deterministic Simulations of Real Websites (NeurIPS 2025 · similarity 0.92)
- Mind2Web 2: Evaluating Agentic Search with Agent-as-a-Judge (NeurIPS 2025 · similarity 0.91)
- AgentHop: A Diagnostic Benchmark for Agentic Multi-Hop Scientific Question Answering (NeurIPS 2026 · similarity 0.89)
- InteractComp: Evaluating Search Agents With Ambiguous Queries (ICML 2026 · similarity 0.88)
- Empowering Efficiency and Efficacy in WebAgent via Enabling Info-Rich Seeking (ICLR 2026 · similarity 0.88)
- WideSearch: Benchmarking Agentic Broad Info-Seeking (ICLR 2026 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.