Research map / Vision-language and multimodal / Multimodal LLMs
Zooming from Context to Cue: Hierarchical Preference Optimization for Multi-Image MLLMs
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes CcDPO, a two-level direct preference optimization framework (context-level captioning preferences with truncation/swap perturbations, and needle-level region-prompted and vision-contrastive preferences) trained on the auto-generated MultiScope-42k dataset to reduce hallucinations and improve multi-image understanding in MLLMs.
Problem
MLLMs struggle with multi-image understanding, suffering context omission, context conflation, and detail misinterpretation; prior multi-image DPO (MIA-DPO) relies on explicit image references and text-only preferences, limiting holistic context modeling.
Approach
Hierarchical DPO: context-level preferences built from per-image structured captions with sequence truncation and content swapping as rejections, plus needle-level preferences using region-marked visual prompts and vision-contrastive image pairs (focus and reject losses), applied in three stages to LLaVA-OV-7B and Qwen2-VL-7B.
Key results
- CcDPO on LLaVA-OV-7B raises average multi-image accuracy across seven benchmarks from 62.5 to 67.1 (+4.6), versus 61.9 for MIA-DPO.
- On Context-AMBER with 8 images, CcDPO on LLaVA-OV-7B lowers CHAIR from 50.6 to 15.3 and raises SCover from 10.3% to 83.3%.
- On MM-NIAH, CcDPO on LLaVA-OV-7B averages 84.5 versus 71.7 for the base model and 81.1 for MIA-DPO.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Apply CcDPO instead of MIA-DPO | LLaVA-OV-7B · MUIRBench | accuracy: 41.4 -> 48.6 | Helped |
| Apply CcDPO instead of MIA-DPO | LLaVA-OV-7B · BLINK | accuracy: 53.7 -> 55.9 | Helped |
| Apply CcDPO instead of MIA-DPO | LLaVA-OV-7B · MIBench | accuracy: 67.8 -> 75.2 | Helped |
| Apply CcDPO instead of MIA-DPO | LLaVA-OV-7B · MM-NIAH average | accuracy: 81.1 -> 84.5 | Helped |
| Apply CcDPO instead of MIA-DPO | LLaVA-OV-7B · Context-AMBER (8 images) | SCover: 36.7% -> 83.3% | Helped |
| Add Needle-Level optimization after Context-Level (ablation) | LLaVA-OV-7B · MUIRBench Overall | accuracy: 44.8 -> 48.6 | Helped |
| Add Needle-Level VDPO after Context-Level and TDPO (ablation) | LLaVA-OV-7B · MUIRBench Similarity | accuracy: 48.0 -> 49.5 | Helped |
| Reduce Context-Level data to 13.6k while keeping Needle-Level data fixed (ablation) | CcDPO · MUIRBench | accuracy: 48.6 -> 46.7 | Hurt |
| Train Context-Level first, then Needle-Level instead of mixing both from the beginning (ablation) | LLaVA-OV-7B · MUIRBench | accuracy: 46.2 -> 48.5 | Helped |
| Train Context-Level first, then Needle-Level instead of mixing both from the beginning (ablation) | LLaVA-OV-7B · Mantis-Eval | accuracy: 65.4 -> 69.6 | Helped |
Surprising findings
- Multi-image DPO gains on single-image tasks, with the paper reporting average +1.2 for LLaVA-OV and +0.3 for Qwen2-VL, contrary to prior reports that multi-image training degrades single-image performance.
- MIA-DPO underperforms its SFT baseline on MuirBench for LLaVA-OV (41.4 vs 45.4).
Limitations
- Does not explicitly model temporal dependencies, so may underperform on video-like inputs.
- OCR supervision is limited in the current dataset.
- Direct comparison with other MLLMs is not fair due to differing base models and data.
Experimental setup
Datasets: LLaVA-23K MDVP MVC COCO AMBER
Benchmarks: MUIRBench MIRB BLINK Mantis-Eval NLVR2 Q-Bench2 MIBench MM-NIAH MMStar ScienceQA MMMU POPE
Metrics: accuracy CHAIR Hal Cog SCover
Baselines: LLaVA-OV Qwen2-VL MIA-DPO SFT GPT-4o InternVL2-8B Mantis-Idefics2
Builds on: DPO MIA-DPO LoRA
Closest papers by meaning
- Task Preference Optimization: Improving Multimodal Large Language Models with Vision Task Alignment (CVPR 2025 · similarity 0.91)
- MIA-DPO: Multi-Image Augmented Direct Preference Optimization For Large Vision-Language Models (ICLR 2025 · similarity 0.91)
- Uncertainty-Aware Exploratory Direct Preference Optimization for Multimodal Large Language Models (CVPR 2026 · similarity 0.91)
- CHiP: Cross-modal Hierarchical Direct Preference Optimization for Multimodal LLMs (ICLR 2025 · similarity 0.90)
- CoVFT: Context-aware Visual Fine-tuning for Multimodal Large Language Models (CVPR 2026 · similarity 0.90)
- Zooming without Zooming: Region-to-Image Distillation for Fine-Grained Multimodal Perception (ICML 2026 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.