Research map / Vision-language and multimodal / Multimodal LLMs

IRIS: Implicit Reward-Guided Internal Sifting for Mitigating Multimodal Hallucination

ICML 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Preference tuning against hallucination · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes IRIS, an iterative on-policy DPO framework that scores self-generated responses with length-normalized implicit rewards (RVG, penalizing text-only-favored candidates) and trains LLaVA-1.5 on 5.7k samples without external feedback.

Problem

Multimodal LLMs hallucinate because language priors override visual evidence; existing DPO-based mitigation relies on costly external evaluators (GPT-4V etc.) that discretize feedback and produce off-policy pairs.

Approach

Uses the model's own implicit DPO rewards (image-conditioned log-ratio minus a rectified text-only log-ratio, RVG) to sift best/worst of K on-policy samples into preference pairs, then optimizes a DPO loss plus a visual-perturbation preference term and an anchor term over iterative rounds.

Key results

• IRIS-R2 on LLaVA-1.5-7B reaches CHAIRs 8.66 on Object HalBench (vs 54.70 base) and AMBER HalRate 11.3 (vs 40.4 base) using 5.7k samples and no external feedback.
• On LLaVA-1.5-13B, IRIS-R2 achieves AMBER CHAIR 3.5 and Object HalBench CHAIRs 10 with 5.7k samples.
• Ablation: RVG at round 2 gives CHAIRs 8.6 vs 14.1 for image-only scoring.

Surprising findings

• Removing the anchored regularization drops LLaVA-Bench overall accuracy below the base model (52.3 vs 55.7), even though hallucination metrics improve.
• Round 3 degrades general capability on LLaVA-Bench (53.2 vs 56.4 at round 2 for 7B).

Limitations

• MMHal-Bench gains are smaller, attributed to compositional reasoning not targeted by the method.
• General capability degrades at round 3 and without the anchor term.
• Efficiency comparison uses different hardware and implementations, so cost ratio is not directly comparable.

Experimental setup

Datasets: RLHF-V

Benchmarks: AMBER MMHal-Bench Object HalBench LLaVA-Bench (in-the-Wild)

Metrics: CHAIR Cover HalRate Cog Score CHAIRs CHAIRi accuracy

Baselines: LLaVA-RLHF HALVA mDPO HA-DPO V-DPO POVID RLAIF-V SymMPO RLHF-V (HD) LPOI OPA-DPO

Builds on: DPO RLAIF-V SeRA DICE mDPO

Closest papers by meaning

• Mitigating Hallucination Through Theory-Consistent Symmetric Multimodal Preference Optimization (NeurIPS 2025 · similarity 0.92)
• Automated Multi-level Preference for MLLMs (NeurIPS 2024 · similarity 0.92)
• Mitigating Hallucinations in Large Vision-Language Models via DPO: On-Policy Data Hold the Key (CVPR 2025 · similarity 0.92)
• Robust Multimodal Large Language Models Against Modality Conflict (ICML 2025 · similarity 0.92)
• BALTO: Balanced Token-Level Policy Optimization for Hallucination Mitigation (NeurIPS 2026 · similarity 0.91)
• CHiP: Cross-modal Hierarchical Direct Preference Optimization for Multimodal LLMs (ICLR 2025 · similarity 0.91)