Research map / Multi-agent systems and games / LLM agents
PatchScout: Thematic Web Data Collection via Information Foraging
Michael West, Eduard Dragut
Read the paperCheck your idea against 137,217 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces PatchScout, a method for thematic web data collection that gathers semantically related documents scattered across disconnected web regions, framed via information foraging.
Problem
Researchers need reusable web corpora of documents sharing a common theme but spread across heterogeneous, structurally disconnected web sources; existing traversal-based and query-based approaches are insufficient for this.
Approach
Frames thematic web data collection as an information foraging problem, so the crawler forages across decentralized web patches for theme-relevant documents rather than following link structure or answering a single query.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add re-seeding (Fixed-Reseed vs No-Reseed) (ablation) | PatchScout · Universities | coverage (%): 1.7 -> 16.1 | Improved |
| add re-seeding (Fixed-Reseed vs No-Reseed) (ablation) | PatchScout · ASPs | coverage (%): 1.5 -> 30.8 | Improved |
| use policy-guided instead of random patch switching (ablation) | PatchScout (Scent-EMA) · Law Schools | coverage (%): 25.4 -> 86.8 | Improved |
| use policy-guided instead of random patch switching (ablation) | PatchScout (Scent-EMA) · ASPs | coverage (%): 9.0 -> 35.3 | Improved |
| generate queries adaptively with feedback instead of upfront (ablation) | PatchScout (Scent-EMA) · Fortune 500 | coverage (%): 6.6 -> 24.6 | Improved |
| use scent-based link ranking instead of random link selection (ablation) | PatchScout (Realized-EMA) · Universities | harvest rate (%): 53.5 -> 86.2 | Improved |
| use scent-based re-seeding instead of realized-productivity re-seeding | PatchScout · Universities | coverage (%): Realized-Mean 2.4 -> Scent-Mean 20.3 | Improved |
| use realized-productivity policy | PatchScout (Realized-Mean) · Universities | harvest rate (%): HR 73.1, coverage 2.4 | Mixed |
| use fixed threshold re-seeding instead of finer-grained scent estimates | PatchScout (Threshold) · ASPs | coverage (%): 35.3 -> 39.1 | Improved |
| PatchScout vs deep-research agents when an authoritative directory exists | PatchScout (best) · Law Schools | coverage (%): Gemini 93.4 vs 86.8 | Hurt |
| PatchScout vs deep-research agents on large target set | PatchScout (Scent-Mean) · Universities | coverage (%): 1.8 -> 20.3 | Improved |
| use smaller Qwen critic instead of DeepSeek critic | PatchScout Critic · 10,000 sampled pages | agreement (%): 86.1, 87.4, 86.9 for 9B, 27B, 397B | No clear effect |
Experimental setup
Baselines: traversal-based methods query-based methods
Closest papers by meaning
- Go-Browse: Training Web Agents with Structured Exploration (ICLR 2026 · similarity 0.87)
- InfoMosaic-Bench: Evaluating Multi-Source Information Seeking in Tool-Augmented Agents (ICLR 2026 · similarity 0.87)
- EcoGEO: Trajectory-Aware Evidence Ecosystems for Web-Enabled LLM Search Agents (NeurIPS 2026 · similarity 0.86)
- Empowering Efficiency and Efficacy in WebAgent via Enabling Info-Rich Seeking (ICLR 2026 · similarity 0.86)
- UIS-Digger: Towards Comprehensive Research Agent Systems for Real-world Unindexed Information Seeking (ICLR 2026 · similarity 0.86)
- AutoData: A Multi-Agent System for Open Web Data Collection (NeurIPS 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
