Research map / Vision-language and multimodal / Multimodal LLMs

HiDe: Rethinking The Zoom-IN method in High Resolution MLLMs via Hierarchical Decoupling

ICML 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Training-free token reduction for MLLMs

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes HiDe, a training-free zoom-in replacement that uses token-wise attention decoupling to localize query-relevant regions and layout-preserving decoupling to build a compact image, boosting Qwen2.5-VL 7B to 92.1% on V*Bench.

Problem

MLLMs perform poorly on high-resolution images; existing zoom-in methods conflate upscaling and cropping, are inefficient (multi-pass or tree search), miss multiple targets, or require costly training.

Approach

Hierarchical decoupling analysis shows background interference, not object size, is the main failure mode. HiDe extracts per-token cross-attention maps for semantic question tokens (minus a search-prompt noise prior), binarizes them into boxes, and recomposes the regions on a compact canvas that preserves their relative spatial layout.

Key results

• HiDe boosts Qwen2.5-VL 7B to 92.1% average on V*Bench (vs 79.1% base and 90.1% DeepEyes) and InternVL3 8B to 91.6%.
• HiDe reduces peak GPU memory from over 96 GB (ViCrop OOM) to about 20 GB and halves inference time versus ViCrop* (about 14 vs 28 min).
• On HRBench-8K, HiDe with Qwen3-VL 8B reaches 83.9% FCP and 81.2%/77.5% average on HRBench-4K for Qwen3-VL 8B and Qwen2.5-VL 7B respectively.

Surprising findings

• Simply upscaling images without cropping does not give stable gains and can hurt multi-object performance.
• Using only the compact cropped image (without the original image) still gives large gains over the base model, though slightly lower than combining both.

Limitations

• Hyperparameters (sigma, alpha, attention layer) are selected per model via grid search on V*Bench, which may overfit to that benchmark despite the claimed robustness.
• Failure cases arise when the attention localizes the wrong target, especially in spatial reasoning over multiple objects.
• Relies on the internal attention of the MLLM, so localization quality depends on the base model's grounding ability.

Experimental setup

Datasets: V*Bench HRBench-4K HRBench-8K POPE MME-RealWorld-Lite MME-RealWorld-English DocVQA COD10K

Benchmarks: V*Bench HRBench-4K HRBench-8K POPE MME-RealWorld-Lite MME-RealWorld-English DocVQA COD10K

Metrics: accuracy IoU precision recall peak GPU memory inference time

Baselines: ViCrop DeepEyes GPT-4o OpenAI o3 Qwen2.5-VL 7B InternVL3 8B

Builds on: ViCrop FlashAttention DeepEyes

Closest papers by meaning

• Zooming without Zooming: Region-to-Image Distillation for Fine-Grained Multimodal Perception (ICML 2026 · similarity 0.89)
• MLLMs Know Where to Look: Training-free Perception of Small Visual Details with Multimodal LLMs (ICLR 2025 · similarity 0.89)
• Seeing What Matters: A Training-Free Self-Guided Framework for Multimodal Detail Perception and Reasoning (CVPR 2026 · similarity 0.89)
• Constructive Distortion: Improving MLLMs with Attention-Guided Image Warping (ICLR 2026 · similarity 0.89)
• FOCUS: Internal MLLM Representations for Efficient Fine-Grained Visual Question Answering (NeurIPS 2025 · similarity 0.89)
• Where Does Vision Meet Language? Understanding and Refining Visual Fusion in MLLMs via Contrastive Attention (CVPR 2026 · similarity 0.89)