Research map / Vision-language and multimodal / Multimodal LLMs

MLLM can see? Dynamic Correction Decoding for Hallucination Mitigation

ICLR 2025 · Vision-language and multimodal › Multimodal LLMs · approach: Decoding-time hallucination mitigation · 3 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Shows MLLMs recognize objects in preceding layers yet suppress them in final output, and proposes DeCo, a training-free decoding method that corrects final-layer logits with preceding-layer knowledge to reduce object hallucination.

Problem

Object hallucination in multimodal LLMs: models generate non-existent objects and omit visible ones, and the internal reasons are poorly understood.

Approach

Probing shows MLLMs encode object existence in preceding layers (roughly 20-28) while LLM priors suppress ground-truth tokens in the final layer; DeCo selects an anchor preceding layer dynamically and adds its logits, scaled by max probability, to the final-layer logits during decoding.

Key results

• On CHAIR with greedy decoding, DeCo lowers LLaVA-1.5 CHAIR_S from 45.0 to 37.8 and CHAIR_I from 14.7 to 11.1.
• On POPE with greedy decoding, DeCo improves MiniGPT-4 F1 from 58.5 to 77.4.
• DeCo adds about 1.2x latency versus vanilla decoding, compared with 1.8x for VCD and 5.1x for OPERA.

Measured effects reported in the paper

Surprising findings

• Ground-truth tokens are often top-ranked in intermediate layers (around layer 18 onward) but fall below hallucinated tokens in the final layer.
• About 91% of hallucinated tokens remain candidates even without the image input, suggesting LLM priors drive them.

Limitations

• Experiments limited to four 7B MLLMs due to GPU cost; generalization to other families and larger sizes untested.
• Slight decrease in detailedness in GPT-4o evaluation; trade-off with truthfulness.
• Layer interval (20-28) and alpha must be tuned empirically for other models.

Experimental setup

Datasets: MSCOCO 2014 AMBER POPE-style probing set (1,200 samples)

Benchmarks: CHAIR POPE MME AMBER MMVet

Metrics: CHAIR_S CHAIR_I F1 accuracy detailedness coherence latency throughput

Baselines: DoLa VCD OPERA Vanilla decoding

Builds on: DoLa VCD OPERA

Closest papers by meaning

• The Mirage of Performance Gains: Why Contrastive Decoding Fails to Mitigate Object Hallucinations in MLLMs? (NeurIPS 2025 · similarity 0.94)
• Look Twice Before You Answer: Memory-Space Visual Retracing for Hallucination Mitigation in Multimodal Large Language Models (ICML 2025 · similarity 0.93)
• Grounding Language with Vision: A Conditional Mutual Information Calibrated Decoding Strategy for Reducing Hallucinations in LVLMs (NeurIPS 2025 · similarity 0.93)
• Self-Correcting Decoding with Generative Feedback for Mitigating Hallucinations in Large Vision-Language Models (ICLR 2025 · similarity 0.93)
• OPERA: Alleviating Hallucination in Multi-Modal Large Language Models via Over-Trust Penalty and Retrospection-Allocation (CVPR 2024 · similarity 0.93)
• ConVis: Contrastive Decoding with Hallucination Visualization for Mitigating Hallucinations in Multimodal Large Language Models (AAAI 2025 · similarity 0.93)