Research map / Vision-language and multimodal / Multimodal LLMs

P$^2$-DPO:Grounding Hallucination in Perceptual Processing via Calibration Direct Preference Optimization

ICLR 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Preference tuning against hallucination · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes P²-DPO, a self-generated on-policy DPO framework that builds focus-and-enhance and visual-robustness preference pairs from the model's own outputs and adds a calibration loss and dynamic deficit-weighting to improve LVLM hallucination resistance without human feedback.

Problem

LVLM hallucination persists even when attention localizes correct regions (perceptual bottleneck) and when images are slightly degraded (lack of visual robustness); existing DPO preference data is vision-agnostic and off-policy, and relies on costly human or AI feedback.

Approach

The model generates its own vision-aware preference pairs: winners come from attention-cropped enhanced inputs or clean images via contrastive amplification, losers from erased or noisy inputs. A calibration loss ties preference to visual dependency, and CLIP-based dynamic deficit-weighting balances the two pair types.

Key results

• On LLaVA-1.5-7B, P²-DPO reaches POPE average F1 87.44 vs 85.10 base and 86.38 for DPO(RLHF-V), with HallusionBench aAcc 55.62 vs 48.16 base.
• On LLaVA-1.5-7B, AMBER relational F1_R improves by +8.5 points over the base model (70.9 vs 62.4).
• P²-DPO improves LLaVA-1.5-7B POPE F1 by over 4 points at Gaussian noise sigma=0.20 relative to the base model (per the robustness stress test).

Surprising findings

• Off-policy RLHF-V pairs show a strongly negative average Implicit Preference Strength (-58.52) with 68.3% negative ratio, while on-policy pairs are positive.

Limitations

• Relies on attention maps and a heuristic adaptive cropping algorithm to locate salient regions
• Several theoretical propositions are stated as assumptions or heuristics rather than proven
• Gains on some benchmarks are marginal or mixed (e.g., Qwen2.5-VL-7B AMBER F1_A decreases)

Experimental setup

Datasets: RLHF-V

Benchmarks: POPE HallusionBench MMHal-Bench AMBER TextVQA

Metrics: F1 accuracy (qAcc, fAcc, aAcc) hallucination rate MMHal score CHAIRi AMBER F1 (E, A, R) Attention Focus Ratio (AFR) Processing Accuracy Implicit Preference Strength (IPS)

Baselines: VCD HA-DPO V-DPO (RLHF-V) V-DPO (SAD) DPO (RLHF-V)

Builds on: DPO mDPO RLHF-V contrastive decoding (VCD) HA-DPO

Closest papers by meaning

• Mitigating Hallucinations in Large Vision-Language Models via DPO: On-Policy Data Hold the Key (CVPR 2025 · similarity 0.94)
• Mitigating Hallucination Through Theory-Consistent Symmetric Multimodal Preference Optimization (NeurIPS 2025 · similarity 0.94)
• Uncertainty-Aware Exploratory Direct Preference Optimization for Multimodal Large Language Models (CVPR 2026 · similarity 0.92)
• SECOND: Mitigating Perceptual Hallucination in Vision-Language Models via Selective and Contrastive Decoding (ICML 2025 · similarity 0.91)
• VES-RFT: Rewarding Visual Evidence Sensitivity to Mitigate Hallucinations in Large Vision-Language Models (CVPR 2026 · similarity 0.91)
• Stop Learning it all to Mitigate Visual Hallucination, Focus on the Hallucination Target. (CVPR 2025 · similarity 0.91)