Research map / Vision-language and multimodal / Multimodal LLMs

Same Attention, Different Truths: Put Logit-Lens over Visual Attention to Detect and Mitigate LVLM Object Hallucination

CVPR 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Preference tuning against hallucination · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Shows that real and hallucinated objects receive similar mid-to-late-layer image attention, uses Logit Lens to detect hallucinations via semantic consistency of attended regions, and mitigates them training-free with region masking (HARM) and visual-evidence-enhanced decoding (VEED).

Problem

Object hallucination in LVLMs is commonly attributed to insufficient visual attention; the paper questions this and asks what the model attends to and why hallucinations occur.

Approach

Real and hallucinated objects get equally strong image attention in mid-to-late layers, but Logit Lens decoding of the attended patches recovers the object token only for real objects. Hallucinations split into visual-uncertainty cases (vanish when attended regions are masked) and contextual-prior cases (persist after masking), each handled by a different training-free remedy.

Key results

• On CHAIR with LLaVA-1.5-7B, the method reaches CHAIR_S 26.8 and CHAIR_I 10.0, versus 29.8 and 13.2 for the best prior method PAI.
• On AMBER with LLaVA-1.5-7B, Hal drops to 14.7 (from 18.4 for PAI) while Cover is 51.2, close to the greedy baseline of 51.0.
• For hallucination detection, LLCC achieves F1 0.7932 versus 0.6842 for SVAR, the best baseline.

Surprising findings

• Real and hallucinated object tokens receive comparable image attention magnitudes, so insufficient visual attention does not explain hallucination in this analysis.
• Roughly 2:1 ratio of visual-uncertainty to contextual-prior hallucinations for LLaVA-1.5-7B.

Limitations

• Detection and mitigation depend on hyperparameters (tau_attn, k, tau_sim, alpha) and the Image-Attention Stage layer range chosen from LLaVA-1.5-7B analysis.
• Analysis and statistics are based on a small sample of 500 images from COCO for LLaVA-1.5-7B.

Experimental setup

Datasets: COCO 2014 val

Benchmarks: CHAIR AMBER

Metrics: CHAIR_S CHAIR_I Precision Recall F1 Cover Hal Cog

Baselines: Greedy Beam Nucleus VCD OPERA DeCo Devils PAI Uncertainty Score InterConf SVAR

Builds on: Logit Lens VCD OPERA PAI

Closest papers by meaning

• Devils in Middle Layers of Large Vision-Language Models: Interpreting, Detecting and Mitigating Object Hallucinations via Attention Lens (CVPR 2025 · similarity 0.94)
• Causally-Grounded Dual-Path Attention Intervention for Object Hallucination Mitigation in LVLMs (AAAI 2026 · similarity 0.94)
• Finding the Correct Visual Evidence Without Forgetting: Mitigating Hallucination in LVLMs via Inter-Layer Visual Attention Discrepancy (ICML 2026 · similarity 0.94)
• Analyzing and Mitigating Object Hallucination: A Training Bias Perspective (AAAI 2026 · similarity 0.94)
• Multi-Object Hallucination in Vision Language Models (NeurIPS 2024 · similarity 0.94)
• Mitigating Object Hallucination in Large Vision-Language Models via False Discovery Controlled Visual Data Splitting (NeurIPS 2026 · similarity 0.93)