Research map / Vision-language and multimodal / Multimodal LLMs
Divide, Conquer and Combine: A Training-Free Framework for High-Resolution Image Perception in Multimodal Large Language Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces HR-Bench (4K/8K) for evaluating MLLM perception of high-resolution images and proposes DC^2, a training-free divide-conquer-combine framework that uses recursive patch captions in a visual memory to improve MLLM HR perception.
Problem
MLLMs struggle to perceive fine details in high-resolution images, yet existing benchmarks top out at 2K resolution and existing enhancement methods require costly visual instruction tuning.
Approach
Recursively split an HR image into quadrants down to the encoder resolution, merge similar patches, caption each patch with the MLLM, keep only objects confirmed across recursion layers, and store them with patch coordinates in a visual memory that is queried at inference to supply textual descriptions alongside the question.
Key results
- On HR-Bench 8K, InternVL-1.5-26B with DC^2 reaches 61.3% average accuracy versus 57.9% baseline (+3.4), and LLaVA-v1.5-13B improves by +2.7.
- On V*, LLaVA-v1.5-7B improves from 46.2 to 57.3 with DC^2 (+11.1).
- The best SOTA MLLM reaches 63% on HR-Bench 8K versus 87% for humans (human average 86.8%).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Apply DC² to InternVL-1.5-26B | InternVL-1.5-26B · HR-Bench 8K | accuracy (Avg.): +3.4 | Helped |
| Apply DC² to LLaVA-v1.6-7B | LLaVA-v1.6-7B · HR-Bench 8K | accuracy (Avg.): +2.1 | Helped |
| Apply DC² to LLaVA-v1.5-13B | LLaVA-v1.5-13B · HR-Bench 8K | accuracy (Avg.): +2.7 | Helped |
| Apply DC² to Yi-VL-6B | Yi-VL-6B · HR-Bench 8K | accuracy (Avg.): +1.1 | Helped |
| Apply DC² to LLaVA-v1.5-7B | LLaVA-v1.5-7B · V* | accuracy: 46.2 -> 57.3 | Helped |
| Apply DC² to LLaVA-v1.5-7B | LLaVA-v1.5-7B · POPE | score: 85.6 -> 86.8 | Helped |
| Remove patch merging (equivalent to adding patch merging) (ablation) | DC² with LLaVA-v1.5-7B · V* | accuracy: 58.3 -> 57.3 | Hurt |
| Add visual memory (ablation) | DC² with LLaVA-v1.5-7B · HR-Bench 8K | accuracy: 36.0 -> 39.5 | Helped |
| Add object filtering (ablation) | DC² with LLaVA-v1.5-7B · HR-Bench 8K | accuracy: 35.0 -> 39.5 | Helped |
| Add recursive cropping (ablation) | DC² with LLaVA-v1.5-7B · HR-Bench 8K | accuracy: 32.5 -> 39.5 | Helped |
Surprising findings
- Adding text descriptions of image regions can compensate for visual information lost by downsampling, performing comparably to preserving key regions at full resolution.
- Performance on fine-grained cross-instance perception tends to decline as recursion depth increases with cropping-based methods, while single-instance perception improves.
Limitations
- Merging similar patches slightly reduces accuracy (theta trade-off), and the method's throughput is lower than some alternatives at the most accurate settings.
- Benchmark is small (200 questions) and relies on GPT-4o for question generation with human review.
- Object extraction and hallucination filtering depend on the MLLM's own captions and an LLM-based extraction step.
Experimental setup
Datasets: DIV8K HR-Bench
Benchmarks: HR-Bench 4K HR-Bench 8K V* POPE MME MMBench
Metrics: accuracy uncertainty score throughput recall@2 mIoU
Baselines: InternVL-1.5-26B LLaVA-v1.6-7B LLaVA-v1.5-13B Yi-VL-6B SEAL Visual Search Gemini 1.5 Flash GPT-4o
Builds on: LLaVA-NeXT SEAL Contriever
Closest papers by meaning
- DenseFusion-1M: Merging Vision Experts for Comprehensive Multimodal Perception (NeurIPS 2024 · similarity 0.92)
- Zooming without Zooming: Region-to-Image Distillation for Fine-Grained Multimodal Perception (ICML 2026 · similarity 0.90)
- Unleashing the Intrinsic Visual Representation Capability of Multimodal Large Language Models (CVPR 2026 · similarity 0.90)
- Q-Bench: A Benchmark for General-Purpose Foundation Models on Low-level Vision (ICLR 2024 · similarity 0.90)
- LION: Empowering Multimodal Large Language Model with Dual-Level Visual Knowledge (CVPR 2024 · similarity 0.90)
- Feast Your Eyes: Mixture-of-Resolution Adaptation for Multimodal Large Language Models (ICLR 2025 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.