Research map / Vision-language and multimodal / Multimodal LLMs

Towards Interpreting Visual Information Processing in Vision-Language Models

ICLR 2025 · Vision-language and multimodal › Multimodal LLMs · approach: Bias-aware multi-image robustness · 3 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Analyzes how LLaVA 1.5's language model processes visual tokens via object-token ablation, logit lens, and attention knockout, finding localized object information that becomes vocabulary-interpretable in mid-to-late layers and is extracted at the last token.

Problem

Understanding how the language model component of adapter-style vision-language models localizes, refines, and integrates object information from visual tokens, which are soft prompts not interpretable as language tokens.

Approach

Applies ablation of object-aligned visual tokens, the logit lens to visual token residuals, and attention knockout to LLaVA's LM to show object information is spatially localized, progressively decodable into vocabulary tokens, and extracted from object tokens to the last position in mid-to-late layers.

Key results

• Ablating object tokens plus one buffer ring drops LLaVA 1.5 generative object identification by 71.79% versus 2.56-20.51% for integrated-gradient baselines at comparable token counts.
• Logit lens finds an average of 23.7% of object patch positions decode to the correct object class at the best layer (average layer 25.7 of 33) in LLaVA 1.5.
• Blocking attention from object tokens to the last token in mid-to-late layers reduces relative correct-token probability to 0.88 (all-layer 0.82), while blocking visual-to-last-row attention has no effect.

Measured effects reported in the paper

Surprising findings

• Logit lens decodes visual token representations into vocabulary tokens even though LLaVA is only fine-tuned on VQA and never trained for next-token prediction on images.
• Blocking visual tokens to the last visual row has no measurable effect, contradicting the hypothesis that the model summarizes image information in that row.

Limitations

• Focuses on LLaVA-type adapter VLMs and may not generalize to other architectures.
• Studies simple object identification rather than multi-step reasoning or open-ended QA.
• VQA analysis uses only 100 manually curated images.

Experimental setup

Datasets: COCO Detection 2017 train ImageNet validation COCO validation

Benchmarks: COCO object identification (generative, polling, VQA)

Metrics: percentage decrease in correct object identifications after relative probability of correct token under attention blocki logit lens token-object correspondence rate

Baselines: Integrated Gradients token ablation random token ablation register token ablation

Builds on: logit lens attention knockout integrated gradients causal tracing register tokens

Closest papers by meaning

• What's in the Image? A Deep-Dive into the Vision of Vision Language Models (CVPR 2025 · similarity 0.93)
• Aligning What Vision-Language Models See and Perceive with Adaptive Information Flow (CVPR 2026 · similarity 0.92)
• Mechanisms of Object Localization in Vision-Language Models (CVPR 2026 · similarity 0.92)
• Reading Images Like Texts: Sequential Image Understanding in Vision-Language Models (ICLR 2026 · similarity 0.92)
• The Narrow Gate: Localized Image-Text Communication in Native Multimodal Models (NeurIPS 2025 · similarity 0.92)
• Synthesize Diagnose and Optimize: Towards Fine-Grained Vision-Language Understanding (CVPR 2024 · similarity 0.91)