Research map / Multi-agent systems and games / LLM agents
AutoData: A Multi-Agent System for Open Web Data Collection
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes AutoData, an eight-agent research/development multi-agent system with an oriented hypergraph cache (OHCache) for automated open web data collection from a natural-language instruction, and a new benchmark Instruct2DS.
Problem
Automating open web data collection from a natural-language instruction is brittle with wrapper-based scrapers and costly with LLM-based scrapers and general multi-agent systems, which suffer from token overhead and unstructured broadcast communication.
Approach
Organizes agents into research and development squads coordinated by a manager, with inter-agent messages modeled as oriented hyperedges (source set to target set) in a message hypergraph, structured by a hyperedge formatter, and bulky artifacts stored in a local cache node referenced by ID instead of being sent in messages.
Key results
- On Instruct2DS, AutoData (GPT-4o) reaches 91.85 F1 on Academic, 96.75 on Stock, and 90.14 on Sport, at 5.58 minutes and $0.57, beating Manus (69.27 / 95.24 / 87.48 at $2.49).
- On HumanEval, Ours + GPT-4o achieves 92.5 pass@1 versus 90.2 for GPT-4o alone and 85.9 for MetaGPT.
- In the picture book case study, AutoData reaches 89.58 accuracy and 98.13 completeness vs. Manus 63.93 and 79.76, at $0.91 versus $1.86 cost.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use AutoData instead of Manus | Open web data collection · Instruct2DS Academic | F1: 69.27 -> 91.85 | Helped |
| Use AutoData instead of Manus | Open web data collection · Instruct2DS Stock | F1: 95.24 -> 96.75 | Helped |
| Use AutoData instead of Manus | Open web data collection · Instruct2DS Sport | F1: 87.48 -> 90.14 | Helped |
| Use AutoData instead of AutoScraper | Web information extraction · Extended SWDE | F1: 76.21 -> 77.44 | Helped |
| Use AutoData instead of Manus | Web information extraction · SWDE | F1: 89.22 -> 89.25 | Helped |
| Use AutoData + GPT-4o instead of GPT-4o alone | HumanEval code generation · HumanEval | Pass@1: 90.2 -> 92.5 | Helped |
| Add the research squad (ablation) | AutoData · Instruct2DS | F1 | Helped |
| Add the development squad (ablation) | AutoData · Instruct2DS | F1 | Helped |
| Add OHCache (ablation) | AutoData · Instruct2DS | F1 | Helped |
| Add the oriented hyperedge formatter (ablation) | AutoData · Instruct2DS | F1 | Helped |
| Add the local cache system (ablation) | AutoData · Instruct2DS | F1 | No clear effect |
| Add the local cache system (ablation) | AutoData · Instruct2DS | Cost | Helped |
Surprising findings
- Human-operated coding assistants (Cline, Cursor) and human experts score below AutoData on Instruct2DS while taking longer.
- Removing the local cache system leaves accuracy nearly unchanged but substantially lowers cost, so its benefit is mainly cost rather than performance.
Limitations
- Depends on proprietary GPT-4/GPT-4o; open-weight DeepSeek R1 underperforms GPT-4o.
- Cannot handle client-side rendered sites, login walls, CAPTCHAs, or anti-bot protections.
- Text-centric only; multimodal sources are not supported.
Experimental setup
Datasets: SWDE Extended SWDE HumanEval
Benchmarks: Instruct2DS SWDE Extended SWDE HumanEval
Metrics: F1 precision recall pass@1 execution precision execution recall time expense (USD) accuracy completeness
Baselines: Manus AutoAgent OpenManus AutoAgents Cline Cursor Human AutoScraper COT Reflexion MetaGPT PaLM
Builds on: ReAct MetaGPT AutoScraper
Closest papers by meaning
- WebDS: An End-to-End Benchmark for Web-based Data Science (ICLR 2026 · similarity 0.86)
- Graph2Eval: Automatic Multimodal Task Generation for Agents via Knowledge Graphs (CVPR 2026 · similarity 0.86)
- CoDA: Agentic Systems for Collaborative Data Visualization (ICLR 2026 · similarity 0.86)
- AutoML-Agent: A Multi-Agent LLM Framework for Full-Pipeline AutoML (ICML 2025 · similarity 0.85)
- Go-Browse: Training Web Agents with Structured Exploration (ICLR 2026 · similarity 0.85)
- AgentOccam: A Simple Yet Strong Baseline for LLM-Based Web Agents (ICLR 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.