Research map / Vision-language and multimodal / Multimodal LLMs

BVS: Bayesian Visual Search with Multimodal Large Language Model for Fine-grained Perception

ICML 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Instruction-tuned multimodal assistants · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes BVS, a training-free Bayesian optimization visual search for MLLMs that uses early-stop attention rollout as a prior, a scale-aware non-stationary GP kernel, and GP-UCB to locate tiny objects in ultra-high-resolution images.

Problem

MLLMs struggle with fine-grained perception of tiny objects in ultra-high-resolution cluttered images; existing visual search methods either scan without priors (inefficient) or rely on static priors without posterior correction.

Approach

Formulates visual search as global Bayesian optimization over a continuous spatial-scale manifold, using an early-stopped attention rollout heatmap as the GP prior mean and a scale-aware non-stationary Matérn-type kernel with GP-UCB acquisition to correct the prior via iterative MLLM-scored observations.

Key results

• On V* Bench with Qwen3-VL-8B, BVS reaches 93.2 overall accuracy, about +3.1 over ZoomEye (90.1).
• On HR-Bench 8K with Qwen3-VL-8B, BVS achieves 79.5 overall, and reaches 93.2 V* accuracy in only 5 search steps vs. 92.7 for Matérn kernel at 15 steps.
• On MMStar with Qwen3VL-8B, BVS gives 66.07 overall vs. 66.20 base (-0.13), with +4.0 on fine-grained perception and -2.0 on logic reasoning.

Surprising findings

• On MMStar, BVS slightly lowers overall accuracy (66.07 vs 66.20) and drops on science & technology (-3.6) and logical reasoning (-2.0).
• Full-layer attention rollout achieves only 12.0% ground-truth coverage, far below the early-stop variant (92.9%).

Limitations

• Relies on a proxy model (Qwen2.5-VL-3B/7B) to compute attention priors instead of the evaluated models, with the claim of no unfair prior knowledge not independently verified.
• Theoretical regret bound depends on assumptions (RKHS residual, MIG rates) and the kernel is only locally approximated as Matérn; the Mercer/PSD proof is a sketch.
• Gains on general-perception MMStar are mixed, with slight overall decline.

Experimental setup

Datasets: V* Bench HR-Bench 4K HR-Bench 8K MMStar

Benchmarks: V* Bench HR-Bench 4K HR-Bench 8K MMStar

Metrics: accuracy coverage rate regret

Baselines: ZoomEye RAP ViCrop DeepEyes Thyme Pixel Reasoner Simple O3 TreeVGR Mini-o3 Qwen3-VL w/ tool Matérn kernel

Builds on: ViCrop ZoomEye Attention Rollout GP-UCB Paciorek and Schervish non-stationary covariance

Closest papers by meaning

• CVSearch: Empowering Multimodal LLMs with Cognitive Visual Search for High-Resolution Image Perception (ICML 2026 · similarity 0.91)
• Divide, Conquer and Combine: A Training-Free Framework for High-Resolution Image Perception in Multimodal Large Language Models (AAAI 2025 · similarity 0.90)
• Seeing What Matters: A Training-Free Self-Guided Framework for Multimodal Detail Perception and Reasoning (CVPR 2026 · similarity 0.90)
• Zooming without Zooming: Region-to-Image Distillation for Fine-Grained Multimodal Perception (ICML 2026 · similarity 0.90)
• Benchmarking Large Vision-Language Models on Fine-Grained Image Tasks: A Comprehensive Evaluation (ICLR 2026 · similarity 0.90)
• CoVFT: Context-aware Visual Fine-tuning for Multimodal Large Language Models (CVPR 2026 · similarity 0.90)