Research map / Vision-language and multimodal / Multimodal LLMs
What's in the Image? A Deep-Dive into the Vision of Vision Language Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Empirical attention-knockout analysis of how VLMs (InternVL2-76B, LLaVA-1.5-7B) move visual information into query/generated tokens, finding query tokens act as global image descriptors, mid-layers drive vision-to-language transfer, and fine details are localized via attention; it also proposes an LLM-as-judge F1 protocol and "Image Re-prompting" using a compressed context.
Problem
VLMs are treated as black boxes, and the internal mechanisms by which visual information flows from image tokens to query and generated text tokens are poorly understood.
Approach
Use attention knockout (blocking image-to-query, image-to-generated, or image-to-others attention, from layer l onward) combined with an LLM-as-a-judge F1 metric and SAM-based localization to reveal that query tokens store global image info, mid-layers mediate visual transfer, and fine details are retrieved spatially; these insights enable a compressed-context re-prompting application.
Key results
- Image-to-generated knockout retains an average F1 of 0.4 on 80 COCO images, while blocking image-to-query drops F1 to zero, showing query tokens hold a compressed image summary.
- Mid-layers (20-40 of InternVL2-76B, about 25% of layers) alone yield F1 in [0.75, 0.81], close to the original model, and attention localization accuracy peaks near 73%.
- Image Re-prompting with a compressed context (about 15x fewer tokens) reduces MME ACC and ACC+ by only 2.8% and 5.3% on average versus naive querying; LLM judge agreed with humans on 95.7% of object-existence judgments.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use compressed context for follow-up questions instead of processing the full image again | InternVL2-76B · MME visual question answering | performance: 15x fewer tokens; 96% of the performance | Mixed |
| Block direct image-token access to generated tokens, leaving indirect access through query tokens (ablation) | InternVL2-76B · COCO images | F1: 0.4 | Hurt |
| Block image-token access to query tokens (ablation) | InternVL2-76B · COCO images | F1: F1 score is zero | Hurt |
| Restrict image-token access to generated tokens to mid-layers 20–40 (ablation) | InternVL2-76B · COCO images | F1: F1 ∈ [0.75, 0.81] | Helped |
| Remove access to image tokens in layers outside 20–40 (ablation) | InternVL2-76B · COCO images | F1 | No clear effect |
| Block image-token attention from layer 40 onward (ablation) | InternVL2-76B · COCO images | F1 | No clear effect |
| Keep only the top 5% of image tokens by attention (ablation) | InternVL2-76B · COCO images | F1: 5% | No clear effect |
| Replace compressed context with the description text alone for follow-up questions | InternVL2-76B · MME Celebrity | performance | Hurt |
| Use query tokens alone or selected image tokens alone instead of their compressed-context combination (ablation) | InternVL2-76B · MME Existence, Count, and Color | performance: K=2% | Helped |
| Use the compressed context instead of the Naive baseline | InternVL2-76B · MME OCR and Count | performance | Helped |
| Use the compressed context rather than full image context | InternVL2-76B · MME ACC and ACC+ | accuracy: 2.8%; 5.3% | Hurt |
| Restrict access to top-k image tokens by attention (ablation) | LLaVA-1.5-7B · COCO images | F1: 5%; 13 tokens; above 0.8 | No clear effect |
Surprising findings
- Generated descriptions remain fairly descriptive (F1 about 0.4) even with no direct access to image tokens, relying only on query tokens.
- The compressed context outperformed the naive per-question baseline on OCR and Count tasks in MME.
Limitations
- Analysis limited to two open VLMs and a single prompt ('describe the image') for the main analysis.
- Evaluation depends on GPT-4 judge and SAM pseudo ground-truth masks, some manually filtered.
- Relies on a small set of about 80 COCO images.
Experimental setup
Datasets: COCO MME
Benchmarks: MME COCO
Metrics: F1 (LLM-judged object overlap) precision recall localization accuracy accuracy ACC+
Baselines: Naive per-question querying Describe-to-LLM (GPT-4)
Builds on: attention knockout logit lens SAM
Closest papers by meaning
- Towards Interpreting Visual Information Processing in Vision-Language Models (ICLR 2025 · similarity 0.93)
- Synthesize Diagnose and Optimize: Towards Fine-Grained Vision-Language Understanding (CVPR 2024 · similarity 0.92)
- Unveiling Visual Perception in Language Models: An Attention Head Analysis Approach (CVPR 2025 · similarity 0.91)
- Reading Images Like Texts: Sequential Image Understanding in Vision-Language Models (ICLR 2026 · similarity 0.91)
- Aligning What Vision-Language Models See and Perceive with Adaptive Information Flow (CVPR 2026 · similarity 0.91)
- Large Vision-Language Models Get Lost in Attention (ICML 2026 · similarity 0.91)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.