Research map / Vision-language and multimodal / Multimodal LLMs
Understanding Information Storage and Transfer in Multi-Modal Large Language Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Extends causal tracing to multimodal LLMs (LLaVA, multimodal Phi-2) via token-ID corruption, finds early MLP/attention storage and sparse visual-token transfer, and proposes MultEdit, a closed-form projection-matrix edit for correcting and inserting facts.
Problem
How factual information is stored in and transferred through multimodal LLMs during visual question answering, which prior causal tracing work (designed for text-only LLMs) does not address.
Approach
MultiModalCausalTrace corrupts visual-constraint token IDs by substituting alternative words (instead of Gaussian noise on many visual tokens) to run causal tracing in MLLMs, combined with attention-contribution analysis of information transfer, and a closed-form projection-matrix edit (MultEdit) targeting early causal MLPs.
Key results
- MultEdit raises average probability of the correct answer on incorrectly answered LLaVA questions from 0.07 to 0.82 with a 1.5% accuracy drop on unrelated questions.
- For long-tailed knowledge insertion, MultEdit reaches editing efficacy of 0.83 and generalization of 0.79 with a 1.4% drop on unrelated questions.
- Mid-layer attention contribution (layers 16-17) predicts answer correctness with AUROC 0.63, while model confidence reaches AUROC 0.76.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Edit an early causal MLP layer with MultEdit | LLaVA-7B · Multimodal Known, incorrect-answer edits | editing efficacy (probability): 0.07 to 0.82 | Helped |
| Edit an early causal MLP layer with MultEdit | LLaVA-7B · Multimodal Known, paraphrased questions | generalization (probability): 0.76 | Helped |
| Edit an early causal MLP layer with MultEdit | LLaVA-7B · Unrelated VQA questions | accuracy: 1.5% | Hurt |
| Edit an early causal MLP layer with MultEdit | LLaVA-7B · Encyclopedia-VQA | editing efficacy (probability): 0.83 | Helped |
| Edit an early causal MLP layer with MultEdit | LLaVA-7B · Encyclopedia-VQA, paraphrased questions | generalization (probability): 0.79 | Helped |
| Edit an early causal MLP layer with MultEdit | LLaVA-7B · Unrelated VQA questions | accuracy: 1.4% | Hurt |
| Use Layer 16 and Layer 17 attention contributions to predict answer correctness | LLaVA · VQA-Constraints Multimodal Known | AUROC: 0.63 | Helped |
| Use model confidence rather than Layer 16–17 attention contributions to predict correctness | LLaVA · VQA-Constraints Multimodal Known | AUROC: 0.63 -> 0.76 | Helped |
| Edit early causal MLP layers instead of middle or later MLP layers (ablation) | MultEdit on LLaVA · Model editing tasks | editing efficacy | Helped |
| Replace visual-constraint token IDs to corrupt the input instead of adding Gaussian noise to visual-token embeddings (ablation) | MultiModalCausalTrace · VQA-Constraints Multimodal Known and Multimodal Movies | recovery of causal states | Helped |
Surprising findings
- In MLLMs, factual information is retrieved from early-layer MLPs (layers 1-4 in LLaVA) rather than the mid-layer MLPs that are causal in LLMs such as LLaMA.
- Standard Gaussian-noise causal tracing fails to recover any causal states in MLLMs even with window size 10.
Limitations
- Findings are limited to factual questions on natural images with short-form answers.
- The work does not map visual tokens to specific concepts in the input image.
- MultEdit could be misused to insert false or harmful information into MLLMs.
Experimental setup
Datasets: OK-VQA WikiMovies Known Encyclopedia-VQA
Benchmarks: VQA-Constraints Encyclopedia-VQA OK-VQA
Metrics: indirect estimation effect editing efficacy generalization specificity VQA accuracy AUROC
Baselines: Fine-tuning (all layers) Fine-tuning with constraints Standard causal tracing with Gaussian noise LLaMA-2 (LLM causal tracing comparison)
Builds on: causal tracing (ROME) attention contribution analysis MEMIT-style closed-form editing
Closest papers by meaning
- Cross-modal Information Flow in Multimodal Large Language Models (CVPR 2025 · similarity 0.94)
- Visual CoT: Advancing Multi-Modal Language Models with a Comprehensive Dataset and Benchmark for Chain-of-Thought Reasoning (NeurIPS 2024 · similarity 0.89)
- Efficient Large Multi-modal Models via Visual Context Compression (NeurIPS 2024 · similarity 0.89)
- Thinking Diffusion: Penalize and Guide Visual-Grounded Reasoning in Diffusion Multimodal Language Models (CVPR 2026 · similarity 0.89)
- LENS: Multi-level Evaluation of Multimodal Reasoning with Large Language Models (ICLR 2026 · similarity 0.88)
- Unveiling the Tapestry of Consistency in Large Vision-Language Models (NeurIPS 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.