Research map / Vision-language and multimodal / Multimodal LLMs

Self-Prophetic Decoding to Unlock Visual Search in LVLMs

ICML 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Bias-aware multi-image robustness · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes SeProD, a training-free plug-and-play decoding method that lets a pre-visual-search LVLM (prophet) propose prophetic tokens that a post-trained visual-search LVLM (search model) accepts under a combined probability threshold, improving visual search and some general VQA benchmarks.

Problem

Visual-search post-training of LVLMs degrades intrinsic single-step capabilities (grounding, OCR, spatial, counting) and multi-step reasoning suffers from interference in long contexts.

Approach

Use the pre-visual-search base LVLM as a 'prophet' that drafts single-step prophetic prefixes in parallel; the post-trained search model accepts draft tokens whose combined probability under both models exceeds a threshold, otherwise resamples from its own distribution, enabling training-free coordination of intrinsic abilities.

Key results

• SeProD improves Mini-o3 on VisualProbe hard/medium/easy by +3.3/+2.5/+2.9 points (3B prophet) and by +4.3/+3.2/+4.6 with a 7B prophet.
• On V* Bench overall, DeepEyes improves from 89.0 to 91.1 and Mini-o3 from 86.3 to 89.2 with SeProD.
• Acceptance rate of prophetic tokens is 74.6%, 74.2%, and 80.7% on VisualProbe hard/medium/easy, with speedups of 1.06x, 1.03x, and 1.07x.

Surprising findings

• The naive approach of feeding pre-training model outputs as textual prompts or overwriting outputs often performs worse than the search model alone.
• Post-training reduces grounding accuracy by 49.3% on RefCOCO validation for Mini-o3 relative to its Qwen2.5-VL-7B base, per the motivating analysis.

Limitations

• Gains are modest (typically 1-5 points) and depend on the prophet and search models sharing a base architecture.
• Failure cases arise from grounding errors in the search model and unclear cropped regions.
• The threshold and alpha choices are hyperparameters, though ablations show robustness across tau values.

Experimental setup

Datasets: V* Bench HR-Bench VisualProbe MME-RealWorld ScienceQA OCRBench CVBench RefCOCO

Benchmarks: V* Bench HR-Bench 4K HR-Bench 8K VisualProbe MME-RealWorld ScienceQA OCRBench CVBench

Metrics: accuracy avg@32 avg@8 acceptance rate speedup

Baselines: Pixel Reasoner DeepEyes Mini-o3 ZoomEye RAP DyFo FOCUS Chain-of-Focus Simple-o3 SEAL naive textual prompting

Builds on: speculative decoding Mini-o3 DeepEyes Pixel Reasoner

Closest papers by meaning

• Rationale-Enhanced Decoding for Multi-modal Chain-of-Thought (CVPR 2026 · similarity 0.90)
• Enhancing Large Vision Language Models with Self-Training on Image Comprehension (NeurIPS 2024 · similarity 0.90)
• See It, Say It, Sorted: An Iterative Training-Free Framework for Visually-Grounded Multimodal Reasoning in LVLMs (CVPR 2026 · similarity 0.90)
• SPARC: Separating Perception And Reasoning Circuits for Test-time Scaling of VLMs (ICML 2026 · similarity 0.90)
• Dual-Latent Memory Routing for Vision-Language Reasoning (ICML 2026 · similarity 0.89)
• When to Think and When to Look: Uncertainty-Guided Lookback (CVPR 2026 · similarity 0.89)