Research map / Vision-language and multimodal / Multimodal LLMs

Mitigating Object Hallucination in Large Vision-Language Models via False Discovery Controlled Visual Data Splitting

NeurIPS 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Decoding-time hallucination mitigation · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes CORAL, a training-free decoding method that splits visual features into symmetric ± perturbed views, computes a mirror statistic on token logits, and selects objects under an estimated FDR threshold to reduce object hallucination in LVLMs.

Problem

Object hallucination in large vision-language models arises from visual uncertainty during decoding, and existing contrastive decoding methods lack principled image-level control of false positives.

Approach

Applies sign-symmetric Gaussian perturbations to visual embeddings to create paired mirror views, uses the Gaussian-mirror statistic on logit differences as a test score, and estimates FDR from the negative tail to pick a data-driven threshold controlling the fraction of hallucinated objects per image.

Key results

• On MSCOCO POPE (adversarial), CORAL reaches 91.02 average accuracy versus 89.31 for the second-best AGLA.
• Averaged over LLaVA-OneVision, Qwen2.5-VL and InternVL3 on MSCOCO random split, CORAL achieves overall FDR 0.0736 and power 90.97 versus 0.0810 and 83.50 for AGLA.
• MMBench on InternVL3-8B rises from 83.4 to 94.5 with CORAL, a reported 11.1-point gain.

Surprising findings

• The MMBench gains (e.g., 11.1 points for InternVL3-8B) are large for a decoding-time method targeting object hallucination.
• The authors state that the conditional sign-flip symmetry is an approximate working assumption not guaranteed by Gaussian perturbations, and that token-level FDR does not automatically imply object-level FDR.

Limitations

• Cannot be applied to black-box APIs that do not expose token-level logits.
• Symmetry assumption is approximate; object-level FDR guarantee not established for multi-token objects or CHAIR-style captions.
• Evaluation focuses on object existence queries; relational and attribute hallucinations are not targeted.

Experimental setup

Datasets: MSCOCO A-OKVQA GQA

Benchmarks: POPE MME MMBench CHAIR

Metrics: FDR power accuracy F1 score MME score MMBench score

Baselines: VCD MARINE AGLA

Builds on: Gaussian mirrors knockoffs Visual Contrastive Decoding

Closest papers by meaning

• Mitigating Object Hallucinations in Large Vision-Language Models through Visual Contrastive Decoding (CVPR 2024 · similarity 0.94)
• On Epistemic Uncertainty of Visual Tokens for Object Hallucinations in Large Vision-Language Models (NeurIPS 2025 · similarity 0.93)
• Same Attention, Different Truths: Put Logit-Lens over Visual Attention to Detect and Mitigate LVLM Object Hallucination (CVPR 2026 · similarity 0.93)
• Causally-Grounded Dual-Path Attention Intervention for Object Hallucination Mitigation in LVLMs (AAAI 2026 · similarity 0.93)
• SECOND: Mitigating Perceptual Hallucination in Vision-Language Models via Selective and Contrastive Decoding (ICML 2025 · similarity 0.93)
• VES-RFT: Rewarding Visual Evidence Sensitivity to Mitigate Hallucinations in Large Vision-Language Models (CVPR 2026 · similarity 0.93)