Research map / Vision-language and multimodal / Multimodal LLMs

Seeing but Not Believing: Probing the Disconnect Between Visual Attention and Answer Correctness in VLMs

ICLR 2026 · Vision-language and multimodal › Multimodal LLMs · approach: VLM capability analysis · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

The paper finds VLMs often attend to correct evidence in deep layers even when answering wrongly ("seeing but not believing") and proposes VEA, a training-free inference-time attention-based image masking method that improves VQA accuracy across LLaVA-NeXT, Qwen2.5-VL, Gemma3, and InternVL3.5.

Problem

VLMs can fail on visual question answering even when the correct visual evidence is present in the image; it is unclear whether this is due to failure to perceive the evidence or failure to use it.

Approach

Identify deep "visual grounding" layers via AUROC against human evidence boxes, aggregate their attention into a patch evidence map with denoising and Gaussian smoothing, and darken non-evidence regions of the image before re-asking the question, requiring no training.

Key results

• VEA averages +5.67 Exact Match points (up to +11.1) and +6.83 Token F1 (up to +17.3) over base models across 8 VLMs, with the lowest average rank (1.12 EM, 1.22 F1) versus AGLA's 2.50 EM rank.
• On evidence attribution (token-level AUROC), VEA reaches 79-85 AUROC across six model variants, outperforming AGLA and VAR.
• On TextVQA with LLaVA-NeXT-7B under 60% noise or 30% random masking, VEA still improves accuracy by +16.4 and +25.8 points.

Surprising findings

• VLMs often attend to the ground-truth evidence in deep layers even when their final answer is wrong, the 'seeing but not believing' phenomenon.
• Shallow layers attend mostly to text, while deep layers sparsely localize evidence, with attention to image tokens increasing with depth.

Limitations

• Requires access to intermediate attention activations, so it does not apply to API-only VLMs.
• Attention may not fully capture evidence localization; gradient or probing signals could complement it.
• Per-model layer profiling is needed and could be costly for large-scale or real-time use; LLaVA-NeXT and InternVL3.5 could not extract attention on 80GB, so Qwen2.5-VL-7B served as a delegate for generating augmented images.

Experimental setup

Datasets: VisualCOT

Benchmarks: TextVQA DocVQA SROIE InfographicsQA

Metrics: Exact Match Token F1 AUROC NDCG@all average rank

Baselines: Base Instructioning CGR VAR AGLA

Builds on: VisualCOT AGLA VAR

Closest papers by meaning

• Evidence-RL: Towards Evidence-intensive Visual Reasoning (NeurIPS 2026 · similarity 0.93)
• Aligning What Vision-Language Models See and Perceive with Adaptive Information Flow (CVPR 2026 · similarity 0.91)
• SPARC: Separating Perception And Reasoning Circuits for Test-time Scaling of VLMs (ICML 2026 · similarity 0.91)
• Look-Back: Implicit Visual Re-focusing in MLLM Reasoning (AAAI 2026 · similarity 0.90)
• From Seeing to Thinking: Decoupling Perception and Reasoning Improves Post-Training of Vision-Language Models (ICML 2026 · similarity 0.90)
• Unveiling the Tapestry of Consistency in Large Vision-Language Models (NeurIPS 2024 · similarity 0.90)