AI Paper Atlas

Research map / Vision-language and multimodal / Multimodal LLMs

Imitating the Truth: Attention-aware Truth-Guided Enhancement for Hallucination Mitigation in Large Vision-Language Models

Hairui Ren, Zixuan Wang, Yibo Yang, He Zhao, Fan Tang, Dandan Guo, Yi Chang

ICLR 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Decoding-time hallucination mitigation

Read the paperCheck your idea against 137,217 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes AGE, an attention-aware truth-guided decoding-time enhancement that contrasts truthful and hallucinatory token dynamics to mitigate hallucinations in LVLMs.

Problem

LVLMs generate content inconsistent with visual evidence (hallucinations); existing mitigation relies on auxiliary modules or coarse decoding-time adjustments that overlook fine-grained differences between truthful and hallucinatory tokens.

Approach

Uses fine-grained attention dynamics to distinguish truthful (real) tokens from hallucinatory ones and guides decoding toward truthful tokens, without relying on auxiliary modules, as the abstract indicates; full method details are not available.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
apply AGE attention imitation at decodingLLaVA-1.5-7B · COCO captioningCHAIR_S: 20.80 -> 16.43Improved
apply AGEMiniGPT-4-7B · COCO captioningCHAIR_S: 30.87 -> 15.62Improved
apply AGEMiniGPT-4-7B · POPEaccuracy: 56.77 -> 73.86Improved
apply AGELLaVA-1.5-13B · COCO captioningCHAIR_I: 12.38 -> 12.55No clear effect
use directional image-attention vector instead of self-multiplicative amplification (ablation)LLaVA-1.5-7B · COCO (128 tokens)CHAIR_S: 43.1 -> 35.4Improved
add text attention intervention on top of image intervention (ablation)LLaVA-1.5-7B · COCO (128 tokens)CHAIR_S: 35.4 -> 31.8Improved
text attention intervention alone (ablation)LLaVA-1.5-7B · COCO (128 tokens)CHAIR_I: 14.2 -> 14.9Hurt
intervene on all middle-stage layers instead of one layer (ablation)LLaVA-1.5-7B · COCO captioningCHAIR_S: 16.43 -> 28.40Hurt
intervene on all late-stage layers instead of layers 30-31 (ablation)LLaVA-1.5-7B · COCO captioningCHAIR_S: 16.43 -> 19.20Hurt
compute delta at final layer rather than intervention layer (ablation)LLaVA-1.5-7B · COCO captioningCHAIR_S: 19.40 -> 16.43Improved
intervene image attention in early layers (1,2) (ablation)LLaVA-1.5-7B · COCO captioningBLEU: 15.93 -> 1.03Hurt
increase sample size M for delta from 5 to 200 (ablation)AGE · COCO captioningCHAIR_SNo clear effect

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy