Research map / Vision-language and multimodal / Multimodal LLMs

Unveiling Visual Perception in Language Models: An Attention Head Analysis Approach

CVPR 2025 · Vision-language and multimodal › Multimodal LLMs · approach: Instruction-tuned multimodal assistants · 1 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Analyzes attention heads in LLaVA-style MLLMs (LLaVA-1.5, VIP-LLaVA, LLaVA-1.6 across Vicuna, Mistral, Llama-3, Phi-3) to identify "visual heads" that attend to image tokens, using attention weight and an entropy-based concentration score, and tests pruning of heads.

Problem

How LLMs inside multimodal LLMs attend to visual tokens during inference, and which attention heads are responsible for processing visual information.

Approach

Identifies a class of visual heads by measuring total attention weight on image tokens and an entropy-based concentration score per head, combines them into a layer-weighted detection score, and uses the resulting head rankings to analyze benchmark consistency and to prune heads.

Key results

• Masking the top 20 early-stage visual heads drops accuracy by roughly 20-40% (e.g., 1.5-7B POPE 85.27 to 28.21 with early top 20 masking, per the table).
• Removing up to 100 late-stage heads causes little degradation, suggesting late-layer pruning is feasible.
• Attention patterns correlate strongly (Pearson > 0.8) within a model family and below 0.1 across families.

Measured effects reported in the paper

Surprising findings

• Late-stage heads can be removed in large numbers (up to 100) with little accuracy loss, contrary to expectations that late layers matter most for output.
• Attention-weight patterns that look consistent on a controlled dataset become inconsistent on MMBench and POPE, which motivated adding concentration.

Limitations

• Head detection score is heuristic, with ad hoc layer weighting terms and constants.
• Analysis covers only a small set of LLaVA-style models and mostly PointQA; generalization to other architectures is not established.
• Some reported drops are inconsistent with the stated results and the paper is sparse on methodological detail.

Experimental setup

Datasets: PointQA MMBench POPE SEED QBench

Benchmarks: PointQA MMBench POPE QBench

Metrics: attention weight on image tokens concentration score (entropy-based) accuracy drop after head masking Pearson correlation Earth Mover's Distance

Baselines: retrieval heads

Builds on: Retrieval heads Logit Lens LLaVA-1.5 VIP-LLaVA

Closest papers by meaning

• Where Does Vision Meet Language? Understanding and Refining Visual Fusion in MLLMs via Contrastive Attention (CVPR 2026 · similarity 0.92)
• Seeing is Understanding: Unlocking Causal Attention into Modality-Mutual Attention for Multimodal LLMs (ICML 2026 · similarity 0.92)
• Head Pursuit: Probing Attention Specialization in Multimodal Transformers (NeurIPS 2025 · similarity 0.92)
• What's in the Image? A Deep-Dive into the Vision of Vision Language Models (CVPR 2025 · similarity 0.91)
• Don't Show Pixels, Show Cues: Unlocking Visual Tool Reasoning in Language Models via Perception Programs (CVPR 2026 · similarity 0.91)
• See What You Are Told: Visual Attention Sink in Large Multimodal Models (ICLR 2025 · similarity 0.91)