Research map / Vision-language and multimodal / Multimodal LLMs

WebWatcher: Breaking New Frontiers of Vision-Language Deep Research Agent

ICLR 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Chain-of-thought multimodal reasoning

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces WebWatcher, a vision-language deep research agent trained via SFT cold start on synthetic multimodal tool-use trajectories plus GRPO, and BrowseComp-VL, a BrowseComp-style multimodal benchmark.

Problem

Multimodal deep research agents are limited: visual agents rely on perception tools without deep textual reasoning, and search agents cannot handle visual or complex multi-tool tasks; there is also no hard, BrowseComp-style multimodal benchmark.

Approach

Generate multi-hop, entity-obfuscated QA from web random walks, convert them to VQA by grounding in web images, synthesize ReAct-style multi-tool trajectories with GPT-4o filtering, then SFT cold start followed by GRPO with format and accuracy rewards.

Key results

• WebWatcher-32B reaches 13.6% on HLE (Avg.), above GPT-4o-based OmniSearch at 9.3%, and the text notes 18.2% in its state-of-the-art claim, an inconsistency within the paper.
• WebWatcher-32B scores 27.0% average on BrowseComp-VL, 58.7% on LiveVQA and 55.3% on MMSearch, outperforming RAG baselines and OmniSearch (16.3% BC-VL avg).
• WebWatcher-32B pass@32 on HLE reaches 41.9%, versus 13.6% at pass@1.

Surprising findings

• Injecting chain-of-thought traces from a larger reasoner into the cold start made the small model unstable, so reasoning traces could not replace SFT cold start in this RL setting.
• Direct-inference frontier models such as GPT-4o and Claude-3.7-Sonnet score below 10% average on HLE, while RAG workflows add only moderate gains.

Limitations

• Training data relies heavily on GPT-4o for QA generation, filtering and trajectory annotation, so quality and cost depend on a proprietary model.
• Reported HLE average numbers are inconsistent within the paper (13.6% in the table vs 18.2% in the text for WebWatcher-32B).

Experimental setup

Datasets: BrowseComp-VL training set SimpleVQA-style long-tail QA InfoSeek VQAv2.0 LogicVista Encyclopedic VQA

Benchmarks: BrowseComp-VL HLE LiveVQA SimpleVQA MMSearch

Metrics: pass@1 accuracy pass@k

Baselines: GPT-4o Gemini-2.5-flash Claude-3.7-Sonnet Qwen-2.5-VL-72B OmniSearch o4-mini Gemini-2.5-Pro

Builds on: ReAct GRPO WebDancer WebSailor Qwen2.5-VL

Closest papers by meaning

• WebThinker: Empowering Large Reasoning Models with Deep Research Capability (NeurIPS 2025 · similarity 0.88)
• Quest: Training Frontier Deep Research Agents with Fully Synthetic Tasks (NeurIPS 2026 · similarity 0.88)
• Multimodal Web Navigation with Instruction-Finetuned Foundation Models (ICLR 2024 · similarity 0.87)
• MM-DeepResearch: A Simple and Effective Multimodal Agentic Search Baseline (ICML 2026 · similarity 0.87)
• MiroEval: Benchmarking Multimodal Deep Research Agents in Process and Outcome (NeurIPS 2026 · similarity 0.87)
• ReFAct: Empowering Multimodal Web Agents with Visual and Context Focusing (CVPR 2026 · similarity 0.87)