Research map / Vision-language and multimodal / Multimodal LLMs
Cross-modal Information Flow in Multimodal Large Language Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Uses attention knockout on LLaVA-series MLLMs over GQA-based VQA to trace how image and question information flow across layers, finding a two-stage visual-to-question integration in lower-middle layers followed by question-to-last-position transfer in middle layers.
Problem
How and where visual and linguistic information interact inside auto-regressive multimodal LLMs to produce answers in VQA remains poorly understood.
Approach
Adapts attention knockout (from Geva et al.) to block attention edges between image, question, and last-position token sets across layer windows, measuring relative change in answer probability to localize cross-modal information flow; also uses logit lens and module knockout to study answer formation.
Key results
- Blocking question-to-image attention in layers ~0-4 reduces answer probability by ~60% on average across six GQA tasks, with a second drop around layer 10 (~21% average).
- Blocking last-position attention to question tokens reduces prediction probability by up to ~30% (ChooseAttr) in middle layers, while blocking image-to-last flow has negligible direct effect.
- Removing image tokens from the last 20 layers of LLaVA-v1.6-13B keeps accuracy intact while cutting inference time by ~40%.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| block attention from question positions to image positions in lower-to-middle layers (ablation) | LLaVA-1.5-13b · GQA VQA tasks | relative change in answer probability: ~60% average reduction in first drop (layers ~0-4) | Hurt |
| block attention from question positions to image positions around layer 10 (ablation) | LLaVA-1.5-13b · GQA VQA tasks | relative change in answer probability: ~21% average reduction (ChooseAttr, ChooseCat, ChooseRel, QueryAttr) | Hurt |
| block last position attending to question positions in middle layers (ablation) | LLaVA-1.5-13b · GQA ChooseAttr | relative change in answer probability: up to ~30% decrease | Hurt |
| block last position attending to its own position (ablation) | LLaVA-1.5-13b · GQA VQA tasks | relative change in answer probability: negligible change | No clear effect |
| block last position attending to image positions (ablation) | LLaVA-1.5-13b · GQA VQA tasks | relative change in answer probability: not significant except slight drop on QueryAttr | No clear effect |
| block question attending to other (non-object) image patches in first stage (ablation) | LLaVA-1.5-13b · GQA VQA tasks | relative change in answer probability: predominant reduction in first stage | Hurt |
| block question attending to object-related image patches in second stage (ablation) | LLaVA-1.5-13b · GQA VQA tasks | relative change in answer probability: dominant in second stage | Hurt |
| removing MLP module (zeroing outputs) in the first layers vs MHAT (ablation) | LLaVA-1.5-13b · GQA VQA tasks, Jaccard similarity of top-10 words at question positions | Jaccard similarity reduction: ~90% reduction for MLP at first layer vs ~60% for MHAT | Hurt |
| remove image tokens in the last 20 layers of LLaVA-v1.6-13B (ablation) | LLaVA-v1.6-13B · GQA six VQA datasets | accuracy and inference time: accuracy intact, inference time reduced by ~40% | Helped |
| replace image with dense captions in LLM (Vicuna) analysis | LLM vicuna-1.5-13b · GQA two tasks | information flow pattern | No clear effect |
| use LLaVA-1.5-7b (smaller) instead of 13b | LLaVA-1.5-7b · GQA VQA tasks | magnitude of first-stage image-to-question flow: smaller first drop than 13b; same layer pattern | No clear effect |
Surprising findings
- Image information does not directly flow to the last position; its influence is mediated through question tokens.
- Blocking the last position from attending to itself has negligible effect on the prediction.
Limitations
- Analysis restricted to LLaVA-series models and six GQA question types with correctly predicted samples only.
- Relies on attention knockout interventions that can be confounded by contextualization effects with larger windows.
Experimental setup
Datasets: GQA validation set OKVQA A-OKVQA Visual Genome dense captions
Benchmarks: GQA (validation subset, 6 question types) OKVQA A-OKVQA
Metrics: relative change in answer probability (%) logit lens answer probability Jaccard similarity of top-10 words
Baselines: Vicuna-v1.5-13b (text-only LLM comparison, dense captions)
Builds on: attention knockout (Geva et al.) logit lens causal tracing
Closest papers by meaning
- Understanding Information Storage and Transfer in Multi-Modal Large Language Models (NeurIPS 2024 · similarity 0.94)
- Cross-Lingual Text-Rich Visual Comprehension: An Information Theory Perspective (AAAI 2025 · similarity 0.91)
- Where Does Vision Meet Language? Understanding and Refining Visual Fusion in MLLMs via Contrastive Attention (CVPR 2026 · similarity 0.91)
- Lifting the Veil on Visual Information Flow in MLLMs: Unlocking Pathways to Faster Inference (CVPR 2025 · similarity 0.90)
- Seeing is Understanding: Unlocking Causal Attention into Modality-Mutual Attention for Multimodal LLMs (ICML 2026 · similarity 0.90)
- Evaluating Cross-Modal Reasoning Ability and Problem Charactaristics with Multimodal Item Response Theory (ICLR 2026 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.