Research map / Vision-language and multimodal / Multimodal LLMs

PAS: Prelim Attention Score for Detecting Object Hallucinations in Large Vision-Language Models

CVPR 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Decoding-time hallucination mitigation · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes Prelim Attention Score (PAS), a training-free attention-based detector that flags object hallucinations in LVLMs when generated object tokens attend heavily to previously generated (prelim) tokens.

Problem

Detecting object hallucinations in large vision-language models in a reference-free, training-free, and efficient manner, using only intrinsic inference signals.

Approach

Hallucinated object tokens depend more on previously generated prelim tokens and less on the image; an information-theoretic MI formulation motivates PAS, which sums attention from prelim tokens to the object token (layer 0, averaged over heads), requiring no extra forward passes.

Key results

• PAS achieves 85.0% average AUROC across LLaVA-1.5-7B, MiniGPT-4, and Shikra on MSCOCO and Pascal VOC, outperforming SVAR (80.3%) and other baselines.
• On LLaVA-1.5-7B MSCOCO, PAS reaches 84.2 AUROC versus 81.5 for SVAR and 56.5 for NLL.
• Layer 0 prelim attention (84.8% avg) outperforms layer 0 image attention (82.1% avg) as a detection signal.

Surprising findings

• Prelim attention is a stronger hallucination signal than image attention, the basis of prior SVAR-style methods.
• Layer 0 (the first layer) gives the best detection, which the authors link to earlier layers gathering information.

Limitations

• Based on original full self-attention; may not hold for efficient attention variants that prevent attending to all token types.
• Only evaluated on object existence hallucination, not relational or attribute hallucination.
• Experiments limited to open-ended image description, not VQA.

Experimental setup

Datasets: MSCOCO val2014 (5,000 sampled images) Pascal VOC val2012 (5,823 images)

Benchmarks: MSCOCO Pascal VOC

Metrics: AUROC

Baselines: NLL Entropy Internal Confidence (IC) GLSim SVAR

Builds on: CHAIR SVAR

Closest papers by meaning

• Mitigating Object Hallucinations in Large Vision-Language Models with Assembly of Global and Local Attention (CVPR 2025 · similarity 0.93)
• Devils in Middle Layers of Large Vision-Language Models: Interpreting, Detecting and Mitigating Object Hallucinations via Attention Lens (CVPR 2025 · similarity 0.93)
• HaloProbe: Bayesian Detection and Mitigation of Object Hallucinations in Vision-Language Models (ICML 2026 · similarity 0.93)
• Same Attention, Different Truths: Put Logit-Lens over Visual Attention to Detect and Mitigate LVLM Object Hallucination (CVPR 2026 · similarity 0.93)
• Detecting and Mitigating Hallucination in Large Vision Language Models via Fine-Grained AI Feedback (AAAI 2025 · similarity 0.92)
• Analyzing and Mitigating Object Hallucination: A Training Bias Perspective (AAAI 2026 · similarity 0.92)