Research map / Vision-language and multimodal / Multimodal LLMs

SenseSearch: Empowering Vision-Language Models with High-Resolution Agentic Search-Reasoning via Reinforcement Learning

CVPR 2026 · Vision-language and multimodal › Multimodal LLMs · approach: VLM capability analysis

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Builds SenseSearch, a Qwen2.5-VL-7B agentic VLM trained with cold-start SFT and RL (BN-GSPO) to coordinate text search, image search and image crop tools, and introduces the HR-MMSearch high-resolution search benchmark.

Problem

VLMs have static knowledge and weak fine-grained analysis of high-resolution images, and existing tool-augmented VLMs use search or cropping tools in isolation rather than coordinating them.

Approach

Combine text search, reverse image search and image crop in one multi-turn RL-trained VLM, and stabilize sequence-level RL with BN-GSPO, which adds a second batch-level normalization of group-normalized advantages on top of GSPO.

Key results

• SenseSearch-RL averages 57.43 across search benchmarks vs 52.49 for MMSearch-R1, and reaches 59.06 on MMSearch.
• SenseSearch-RL scores 83.8 on V* Bench and 73.6 on HR-Bench 4K, above DeepEyes (83.3 and 73.2) and Pixel-Reasoner (84.3 and 72.6 on V*/HR-4K respectively in the table).
• In the BN-GSPO ablation, BN-GSPO reaches 56.72 MMSearch and 79.05 V* Bench, above GRPO (50.88, 67.54) and GSPO (53.80, 53.93).

Surprising findings

• GSPO without batch normalization collapses on HR-Bench 4K (44.50) and V* Bench (53.93) relative to its MMSearch gain, suggesting reward-scale variance hurts multi-tool RL.

Limitations

• Relies on GPT-4o as LLM judge for both rewards and evaluation.
• HR-MMSearch is small (305 images) and questions are manually crafted.
• Search results are cached and pre-fetched for RL training, which may not reflect live web conditions.

Experimental setup

Datasets: FVQA-train Pixel-Reasoner warm-start corpus VisualProbe-train DeepEyes-4K-train

Benchmarks: FVQA-test InfoSeek MMSearch SimpleVQA LiveVQA MAT-Search HR-MMSearch V* Bench HR-Bench MME RealWorld

Metrics: Pass@1 Avg@8 exact match exact match

Baselines: MMSearch-R1 DeepMMSearch-R1 Visual-ARFT Pixel-Reasoner DeepEyes GRPO GSPO GPT-4o Gemini-2.5-Flash GPT-4o-mini

Builds on: GSPO MMSearch-R1 DeepEyes Pixel-Reasoner DAPO

Closest papers by meaning

• Agentic Jigsaw Interaction Learning for Enhancing Visual Perception and Reasoning in Vision-Language Models (ICLR 2026 · similarity 0.91)
• CodeV: Code with Images for Faithful Visual Reasoning via Tool-Aware Policy Optimization (CVPR 2026 · similarity 0.90)
• Agent0-VL: Exploring Self-Evolving Agent for Tool-Integrated Vision-Language Reasoning (ICML 2026 · similarity 0.90)
• VRAG-RL: Empower Vision-Perception-Based RAG for Visually Rich Information Understanding via Iterative Reasoning with Reinforcement Learning (NeurIPS 2025 · similarity 0.90)
• Scaling Agentic Reinforcement Learning for Tool-Integrated Reasoning in VLMs (CVPR 2026 · similarity 0.90)
• OpenSearch-VL: An Open Recipe for Frontier Multimodal Search Agents (NeurIPS 2026 · similarity 0.90)