Research map / Multi-agent systems and games / LLM agents
UIS-Digger: Towards Comprehensive Research Agent Systems for Real-world Unindexed Information Seeking
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces the unindexed information seeking (UIS) problem, a 110-item expert-annotated benchmark UIS-QA, and UIS-Digger, a multi-agent browsing/file-parsing system whose ~30B backbone is tuned with SFT and rejection-sampling fine-tuning (RFT).
Problem
Information-seeking agents rely on search-engine-indexed content and fail on information that is not indexed (overlooked pages, dynamic pages, embedded files), and existing benchmarks do not measure this capability.
Approach
Defines unindexed information formally, builds a benchmark of UIS questions filtered to exclude indexed-answerable ones, and proposes a multi-agent system with a dual-mode (textual/visual) web surfer sharing memory, a file reader, and parallel tool calls, trained on synthetic real-web and simulated-site QA pairs via SFT then RFT.
Key results
- UIS-Digger reaches 27.3% accuracy on UIS-QA with both Pangu-38B and Qwen3-32B backbones, the highest reported in the paper.
- Strong baselines drop sharply on UIS-QA: Tongyi-DR scores 70.9 on GAIA-text but 23.6 on UIS-QA; Memento scores 25.5 on UIS-QA.
- For Pangu-38B, SFT adds 13.6 points on UIS-QA (9.1 to 22.7) and RFT adds a further 4.6 points (to 27.3).
Surprising findings
- Untuned O3 inside the framework scores 30.9%, above the tuned Pangu and Qwen3 models, suggesting raw foundation model capability matters greatly.
- GPT-4o as the main LLM scores only 8.2%, showing framework compatibility strongly affects results.
Limitations
- Gains from RFT are limited, possibly due to distribution gap between synthetic and real QA pairs.
- Sparse supervision from final-answer-only rejection sampling may reinforce low-quality trajectories.
- Time-varying web accessibility can turn UIS cases into IIS cases, changing difficulty.
Experimental setup
Datasets: UIS-QA synthetic QA pairs from real websites synthetic QA pairs from simulated websites
Benchmarks: UIS-QA GAIA BrowseComp-zh FinSearchComp
Metrics: accuracy
Baselines: DeepSeek-V3.1 Claude-sonnet-4 GPT-5 GLM-4.5 Doubao Gemini-2.5-pro WebSailor Tongyi DeepResearch DDv2 OWL MiroThinker Memento
Builds on: ReAct Search-R1 WebSailor OWL Memento
Closest papers by meaning
- InfoMosaic-Bench: Evaluating Multi-Source Information Seeking in Tool-Augmented Agents (ICLR 2026 · similarity 0.92)
- Empowering Efficiency and Efficacy in WebAgent via Enabling Info-Rich Seeking (ICLR 2026 · similarity 0.90)
- MindSearch: Mimicking Human Minds Elicits Deep AI Searcher (ICLR 2025 · similarity 0.88)
- EcoGEO: Trajectory-Aware Evidence Ecosystems for Web-Enabled LLM Search Agents (NeurIPS 2026 · similarity 0.88)
- WideSearch: Benchmarking Agentic Broad Info-Seeking (ICLR 2026 · similarity 0.88)
- InteractComp: Evaluating Search Agents With Ambiguous Queries (ICML 2026 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.