Research map / Multi-agent systems and games / LLM agents
LiveResearchBench: Benchmarking Single- and Multi-Agent Systems for Citation-Grounded Deep Research
Jiayu Wang, Yifei Ming, Riya Dulepet, Qinglin Chen, Austin Xu, Zixuan Ke, Frederic Sala, Aws Albarghouthi, Caiming Xiong, Shafiq Joty
Read the paperCheck your idea against 137,217 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces LiveResearchBench, a benchmark of user-centric, dynamic, unambiguous deep-research tasks for evaluating single- and multi-agent systems that produce citation-backed reports from live web search.
Problem
Evaluating deep research agents that produce comprehensive, citation-backed reports by searching hundreds of live websites lacks rigorous benchmarks satisfying user-centricity, dynamism, and unambiguity.
Approach
A benchmark whose tasks are built around realistic user information needs, require up-to-date live web information beyond parametric knowledge, and are phrased unambiguously so that outputs can be consistently judged; it is used to benchmark both single- and multi-agent deep research systems.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use multi-agent deep research systems instead of single-agent systems | deep research systems · LiveResearchBench | average DeepEval score: 62.8 (single web) -> 69.5 (multi-agent) | Improved |
| use multi-agent system for citation association | deep research systems · LiveResearchBench | Citation Association: 61.9 avg multi-agent; Deerflow+ 77.0 vs o3 Deep Research 25.6 | Improved |
| use single-agent deep research (longer reasoning) rather than web-search agent | deep research systems · LiveResearchBench | Citation Association: GPT-5 67.6 -> o3 Deep Research 25.6 | Hurt |
| use multi-agent systems for coverage | Deerflow+ / Open Deep Research · LiveResearchBench | Coverage & Comprehensiveness: GPT-5 83.4 vs Deerflow+ 61.6, ODR 65.3 | Hurt |
| longer reports | deep research systems · LiveResearchBench | report quality | No clear effect |
| use multi-judge ensemble (Gemini 2.5 Pro + GPT-5) instead of direct 0-10 rating (ablation) | DeepEval · human agreement | agreement with humans: <60% (direct rating) -> 98.3% (checklist, ensemble) | Improved |
| use checklist protocol for presentation (ablation) | DeepEval · Presentation & Organization | human agreement (%): 98.3 | Improved |
Experimental setup
Benchmarks: LiveResearchBench
Closest papers by meaning
- SciResearchBench: Benchmarking AI Agents on Complex Scientific Literature Discovery (NeurIPS 2026 · similarity 0.91)
- AstaBench: Rigorous Benchmarking of AI Agents with a Scientific Research Suite (ICLR 2026 · similarity 0.89)
- MiroEval: Benchmarking Multimodal Deep Research Agents in Process and Outcome (NeurIPS 2026 · similarity 0.89)
- REAL: Benchmarking Autonomous Agents on Deterministic Simulations of Real Websites (NeurIPS 2025 · similarity 0.88)
- Mind2Web 2: Evaluating Agentic Search with Agent-as-a-Judge (NeurIPS 2025 · similarity 0.87)
- Agent-Native Research Artifacts (NeurIPS 2026 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
