Research map / Vision-language and multimodal / Multimodal LLMs
NAVIGATE: Evaluating Visual-Guided Search Decision-Making on the Open Web
YaoQi Fan, Zhe Chen, Zhu Wei, Kangxin Yin, Yangzhou Liu, Yue Cao, Zhi Zhu, Tong Lu
Read the paperCheck your idea against 137,217 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces NAVIGATE, a benchmark for evaluating vision-language models on deciding when to search and how to steer open-web search from ambiguous or conflicting visual evidence.
Problem
VLMs deployed with web search tools lack benchmarks isolating whether they decide when to search and how to steer search from ambiguous visual evidence, especially with multiple overlapping or conflicting images.
Approach
A benchmark centered on images as primary evidence for open-web search planning, targeting search-decision capability under visual ambiguity and cross-image conflicts.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| enable web search for GPT-5 | GPT-5 · NAVIGATE overall | accuracy: 27.2% -> 33.3% | Improved |
| enable web search for GPT-5.1 | GPT-5.1 · NAVIGATE overall | accuracy: 28.42% -> 34.11% | Improved |
| enable search for Gemini-2.5 family (Flash-Search vs Pro without search) | Gemini-2.5 · NAVIGATE overall | accuracy: 24.27% vs 25.10% | No clear effect |
| enable web search on multi-domain questions (GPT-5) | GPT-5 · NAVIGATE multi-domain questions | accuracy: 16.42% -> 23.88% | Improved |
| scale Qwen2.5-VL from 7B to 72B | Qwen2.5-VL-Instruct · NAVIGATE overall | accuracy: 10.88% -> 15.90% | Improved |
| use thinking variant instead of instruct for Qwen3-VL-235B | Qwen3-VL-235B-A22B · NAVIGATE overall | accuracy: 16.84% -> 17.89% | No clear effect |
| add search to models on single-image questions | search-enabled VLMs · NAVIGATE single-image questions | accuracy | Hurt |
| larger scale or stronger general multimodal ability | VLMs · NAVIGATE overall | accuracy | No clear effect |
Experimental setup
Benchmarks: NAVIGATE
Closest papers by meaning
- V?: Guided Visual Search as a Core Mechanism in Multimodal LLMs (CVPR 2024 · similarity 0.88)
- MMSearch-Plus: Benchmarking Provenance-Aware Search for Multimodal Browsing Agents (ICLR 2026 · similarity 0.87)
- ARK: A Dual-Axis Multimodal Retrieval Benchmark along Reasoning and Knowledge (NeurIPS 2026 · similarity 0.87)
- Diversity Over Frequency: Rethinking Tool Use in Visual Chain-of-Thought Agents (ICML 2026 · similarity 0.87)
- CVSearch: Empowering Multimodal LLMs with Cognitive Visual Search for High-Resolution Image Perception (ICML 2026 · similarity 0.87)
- FlySearch: Exploring how vision-language models explore (NeurIPS 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
