Research map / Vision-language and multimodal / Multimodal LLMs
MIA-DPO: Multi-Image Augmented Direct Preference Optimization For Large Vision-Language Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes MIA-DPO, a low-cost multi-image visual preference alignment method that builds multi-image prompts from single-image data (sequence, grid collage, pic-in-pic) and selects rejected responses via attention ratios, with no human/API annotation.
Problem
Visual preference alignment (DPO) has been developed for single-image LVLMs, while multi-image settings lack diverse training prompts and cheap chosen/rejected pair construction.
Approach
Extend single-image LLaVA-665k data into multi-image prompts (sequence, grid collage, pic-in-pic) with unrelated distractor images, and use the model's attention ratio on the target image to mark responses that look at wrong images as rejected, avoiding human or GPT annotation.
Key results
- MIA-DPO on LLaVA-v1.5 raises five-benchmark multi-image average from 40.4 to 43.4 (+3.0), with BLINK +5.8.
- MIA-DPO on InternLM-XC2.5 raises the multi-image average from 53.6 to 57.9 (+4.3), with Mantis +11.1.
- On single-image benchmarks, MIA-DPO on LLaVA-v1.5 averages 51.7 vs 51.6 baseline, while on InternLM-XC2.5 it drops from 74.2 to 73.5.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| apply MIA-DPO | LLaVA-v1.5 · five multi-image benchmarks (average) | score: 40.4 -> 43.4; +3.0 | Helped |
| apply MIA-DPO | InternLM-XC2.5 · five multi-image benchmarks (average) | score: 53.6 -> 57.9; +4.3 | Helped |
| apply MIA-DPO | LLaVA-v1.5 · seven single-image benchmarks (average) | score: 51.6 -> 51.7 | Helped |
| apply MIA-DPO | InternLM-XC2.5 · seven single-image benchmarks (average) | score: 74.2 -> 73.5 | Hurt |
| add post-selection (ablation) | MIA-DPO on LLaVA-v1.5 · five multi-image benchmarks (average) | score: 42.3 -> 43.4 | Helped |
| combine Sequence, Grid Collage, and Pic-in-Pic data instead of using one type alone (ablation) | MIA-DPO on LLaVA-v1.5 · five multi-image benchmarks (average) | score: 42.6, 42.4, and 42.7 -> 43.4 | Helped |
| increase gamma from 0.1 to 0.2 (ablation) | MIA-DPO on LLaVA-v1.5 · five multi-image benchmarks (average) | score: 43.3 -> 42.0 | Hurt |
| increase gamma from 0.1 to 0.3 (ablation) | MIA-DPO on LLaVA-v1.5 · five multi-image benchmarks (average) | score: 43.3 -> 42.0 | Hurt |
| train for 2 epochs instead of 3 (ablation) | MIA-DPO on LLaVA-v1.5 · five multi-image benchmarks (average) | score: 43.4 -> 42.5 | Hurt |
| train for 1 epoch instead of 3 (ablation) | MIA-DPO on LLaVA-v1.5 · five multi-image benchmarks (average) | score: 43.4 -> 43.3 | No clear effect |
| use attention-aware data selection instead of GPT-4o-mini selection (ablation) | MIA-DPO on LLaVA-v1.5 · five multi-image benchmarks (average) | score: 42.7 -> 43.4; +0.7 | Helped |
Surprising findings
- Single-image DPO baselines such as POVID drop sharply on multi-image benchmarks (e.g., NLVR2 21.4 vs 52.1 baseline).
- MIA-DPO on InternLM-XC2.5 slightly reduces average single-image performance (74.2 to 73.5).
Limitations
- Post-selection discards about 5% of data and relies on hand-set thresholds per image count.
- Slight single-image degradation on InternLM-XC2.5.
- Attention-ratio selection assumes attention indicates correctness, which is not formally validated.
Experimental setup
Datasets: LLaVA-665k TextVQA COCO GQA OCRVQA
Benchmarks: MMMU BLINK Mantis-Eval NLVR2 MVBench MMStar ScienceQA MM-Vet POPE MMBench MathVista AI2D
Metrics: accuracy average accuracy
Baselines: LLaVA-RLHF HA-DPO POVID GPT-4o-mini selection LLaVA-v1.6 Idefics2 GPT-4V
Builds on: DPO RLHF-V HA-DPO POVID LLaVA-665k
Closest papers by meaning
- Zooming from Context to Cue: Hierarchical Preference Optimization for Multi-Image MLLMs (NeurIPS 2025 · similarity 0.91)
- Task Preference Optimization: Improving Multimodal Large Language Models with Vision Task Alignment (CVPR 2025 · similarity 0.91)
- Fine-Grained Verifiers: Preference Modeling as Next-token Prediction in Vision-Language Alignment (ICLR 2025 · similarity 0.90)
- Importance Sampling for Multi-Negative Multimodal Direct Preference Optimization (ICLR 2026 · similarity 0.89)
- Uncertainty-Aware Exploratory Direct Preference Optimization for Multimodal Large Language Models (CVPR 2026 · similarity 0.89)
- Multimodal Large Language Models Make Text-to-Image Generative Models Align Better (NeurIPS 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.