Research map / Vision-language and multimodal / Multimodal LLMs
Mitigating Hallucination Through Theory-Consistent Symmetric Multimodal Preference Optimization
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SymMPO, a symmetric DPO variant for MLLM hallucination mitigation that adds preferred responses for CLIP-nearest-neighbor contrastive images and a margin consistency loss, evaluated on LLaVA-1.5 across five benchmarks.
Problem
Vision-oriented DPO methods for reducing hallucination in MLLMs use contrastive images with fixed responses, which the paper argues yields a non-rigorous objective (partition functions do not cancel) and indirect preference supervision.
Approach
Symmetric pairwise preference learning: for an image m and its CLIP nearest neighbor m', each gets its own preferred response (y_w, y_w') and the model is trained to rank each response over the other under its own image, plus a margin-consistency regularizer that matches preference margins across the two images and an anchored preference term.
Key results
- On LLaVA-1.5-7B, SymMPO reaches 44.28 aAcc on HallusionBench and 2.89 MMHal score, versus 42.78 aAcc and 2.71 for mDPO under the same data.
- On LLaVA-1.5-13B, SymMPO achieves 3.01 MMHal score and 39.6% MMHal hallucination rate, beating mDPO (2.93, 43.8%).
- SymMPO underperforms DPO on Object-HalBench response-level hallucination (19.5 vs 12.9 for DPO on 7B), which the authors attribute to noisy caption-based preference data.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use SymMPO instead of mDPO | LLaVA-1.5-7B · HallusionBench | aAcc: 42.78 -> 44.28 | Helped |
| use SymMPO instead of mDPO | LLaVA-1.5-13B · HallusionBench | aAcc: 39.85 -> 44.55 | Helped |
| use SymMPO instead of DPO | LLaVA-1.5-7B · Object-HalBench | response-level hallucination rate: 12.9 -> 19.5 | Hurt |
| use SymMPO instead of DPO | LLaVA-1.5-13B · Object-HalBench | response-level hallucination rate: 15.4 -> 20.4 | Hurt |
| use SymMPO instead of mDPO | LLaVA-1.5-7B · MMHal-Bench | Score: 2.71 -> 2.89 | Helped |
| use SymMPO instead of mDPO | LLaVA-1.5-13B · MMHal-Bench | Score: 2.93 -> 3.01 | Helped |
| use SymMPO instead of mDPO | LLaVA-1.5-7B · AMBER | Acc: 80.6 -> 82.6 | Helped |
| use SymMPO instead of mDPO | LLaVA-1.5-13B · AMBER | Acc: 83.8 -> 84.9 | Helped |
| use SymMPO instead of mDPO | LLaVA-1.5-7B · MMStar | Overall: 34.2 -> 34.8 | Helped |
| use SymMPO instead of mDPO | LLaVA-1.5-13B · MMStar | Overall: 35.0 -> 35.2 | Helped |
| add pair-wise preference loss (ablation) | SymMPO on LLaVA-1.5-7B · HallusionBench | aAcc: 43.22 -> 44.28 | Helped |
| add margin consistency regularization (ablation) | SymMPO on LLaVA-1.5-7B · HallusionBench | aAcc: 44.46 -> 44.28 | No clear effect |
Surprising findings
- SymMPO and mDPO are worse than plain DPO on Object-HalBench, which the authors attribute to caption-induced noise in fine-grained preference pairs.
- Black (uninformative) contrastive images degrade SymMPO below mDPO, while other contrastive image types help.
Limitations
- Performance on fine-grained detailed description tasks (Object-HalBench) remains limited due to caption-based data noise.
- Additional computational overhead from generating preferred responses for each contrastive image.
- Hyperparameters tuned by grid search on the evaluation-related setting; gamma is very sensitive.
Experimental setup
Datasets: TPO 21.4k image-prompt pairs (VQA v2, MSCOCO, TextVQA) HallusionBench Object-HalBench MMHal-Bench AMBER MMStar
Benchmarks: HallusionBench Object-HalBench MMHal-Bench AMBER MMStar
Metrics: qAcc fAcc aAcc response-level hallucination rate mention-level hallucination rate MMHal score MMHal hallucination rate accuracy F1 overall score
Baselines: LLaVA-RLHF RLHF-V POVID HALVA HA-DPO RLAIF-V TPO OPA-DPO HSA-DPO DPO mDPO
Builds on: DPO mDPO TPO AncPO RLHF-V
Closest papers by meaning
- P$^2$-DPO:Grounding Hallucination in Perceptual Processing via Calibration Direct Preference Optimization (ICLR 2026 · similarity 0.94)
- Mitigating Hallucinations in Large Vision-Language Models via DPO: On-Policy Data Hold the Key (CVPR 2025 · similarity 0.94)
- CHiP: Cross-modal Hierarchical Direct Preference Optimization for Multimodal LLMs (ICLR 2025 · similarity 0.93)
- Stop Learning it all to Mitigate Visual Hallucination, Focus on the Hallucination Target. (CVPR 2025 · similarity 0.93)
- Automated Multi-level Preference for MLLMs (NeurIPS 2024 · similarity 0.93)
- Uncertainty-Aware Exploratory Direct Preference Optimization for Multimodal Large Language Models (CVPR 2026 · similarity 0.93)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.