Research map / Vision-language and multimodal / Multimodal LLMs
Context Blindness in DPO: Mitigating Object Hallucination in MLLMs via Context-Calibrated Preference Optimization
Byungoh Ko, Jinyoung Park, Jongha Kim, Jeehye Na, Jaewon Cho, Hyunwoo Kim
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add context-calibrated loss (C2-DPO) over C-DPO | Qwen2-VL-Instruct-2B · Object HalBench response | hallucination rate: 2.5 -> 1.6 | Improved |
| add context-calibrated loss over C-DPO | LLaVA-v1.5-7B · Object HalBench response | hallucination rate: 5.9 -> 4.8 | Improved |
| add context-calibrated loss over C-DPO | LLaVA-v1.5-7B · AMBER | Hallucination Score: 14.9 -> 13.8 | Improved |
| replace C-DPO with C2-DPO | Qwen2-VL-Instruct-2B · MM-Vet | overall: 48.2 -> 49.1 (base 49.9) | Mixed |
| apply C2-DPO | LLaVA-v1.5-7B · ScienceQA / TextVQA | accuracy: TextVQA 58.2 -> 58.2 | No clear effect |
| add L_c and DPO(x') jointly to DPO(x) (ablation) | C2-DPO · Object HalBench response (LLaVA) | hallucination rate: 5.9 -> 4.8 | Improved |
| add DPO on degraded context only (ablation) | C2-DPO · Object HalBench response (LLaVA) | hallucination rate: 5.9 -> 7.1 | Hurt |
| add L_c only (no DPO(x')) (ablation) | C2-DPO · Object HalBench response (Qwen) | hallucination rate: 2.5 -> 3.5 | Hurt |
| add calibration to SimPO (ablation) | SimPO · Object HalBench response | hallucination rate: 3.9 -> 3.3 | Improved |
| add calibration to RDPO (ablation) | RDPO · AMBER | Hallucination Score: 38.7 -> 29.0 | Improved |
| add calibration to DPO in text-only setting (ablation) | DPO · AlpacaEval 2 LC | win rate: 24.1 -> 25.9 | Improved |
| degrade by removing caption rather than image (ablation) | C2-DPO · AMBER | Hallucination Score: 18.2 -> 16.1 | Improved |
Closest papers by meaning
- CHiP: Cross-modal Hierarchical Direct Preference Optimization for Multimodal LLMs (ICLR 2025 · similarity 0.92)
- P$^2$-DPO:Grounding Hallucination in Perceptual Processing via Calibration Direct Preference Optimization (ICLR 2026 · similarity 0.92)
- Uncertainty-Aware Exploratory Direct Preference Optimization for Multimodal Large Language Models (CVPR 2026 · similarity 0.92)
- Mitigating Hallucinations in Large Vision-Language Models via DPO: On-Policy Data Hold the Key (CVPR 2025 · similarity 0.92)
- Mitigating Object Hallucination in MLLMs via Data-augmented Phrase-level Alignment (ICLR 2025 · similarity 0.92)
- Mitigating Hallucination Through Theory-Consistent Symmetric Multimodal Preference Optimization (NeurIPS 2025 · similarity 0.91)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
