Research map / Vision-language and multimodal / Multimodal LLMs
Towards Neuron Attributions in Multi-Modal Large Language Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes NAM, a training-free neuron attribution method for multimodal LLMs that attributes generated images (via segmentation and diffusion attribution) and text to FFN neurons using activation-based scores, and uses the I-neurons for image editing.
Problem
Neuron attribution has been developed for text-only LLMs, but extending it to MLLMs that generate images and text is challenging due to semantic noise in generated images, the cost of gradient/causal methods, and the entanglement of text-related (T) and image-related (I) neurons.
Approach
Attribute generated image regions (segmented with EVA02 and mapped via Diffusers-Interpret) to the last hidden representation, then to FFN neurons with a score combining residual-stream contribution and activation magnitude via a max over normalized scores, avoiding gradients; separate modality-specific T- and I-neurons are identified and used for training-free image editing.
Key results
- NAM achieves the highest CLIPScore w.r.t. input images for T-neurons (0.562) and I-neurons (0.558) in GILL, outperforming the best baseline AcT (0.556) and AcU (0.543) on the same metric (text-side and image-side consistency).
- NAM outperforms all baselines by an average of 16.83% in cross-sample invariance of identified neurons.
- NAM-based image editing requires perturbations about 40.2% smaller than baselines on average and nearly 15% smaller than the best baseline.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Combine neuron activation with mapped values on the target semantic dimension rather than using activation alone (ablation) | NAM T-neuron attribution on GILL · COCO | BLEURT: 0.139±0.013 -> 0.148±0.013 | Helped |
| Combine neuron activation with mapped values on the target semantic dimension rather than using mapped values alone (ablation) | NAM T-neuron attribution on GILL · COCO | CLIPScore (input images): 0.543±0.051 -> 0.562±0.054 | Helped |
| Combine neuron activation with mapped values on the target semantic dimension rather than using activation alone (ablation) | NAM I-neuron attribution on GILL · COCO | BLEURT: 0.128±0.013 -> 0.144±0.014 | Helped |
| Combine neuron activation with mapped values on the target semantic dimension rather than using mapped values alone (ablation) | NAM I-neuron attribution on GILL · COCO | CLIPScore (input images): 0.501±0.061 -> 0.558±0.053 | Helped |
| Use NAM-identified I-neurons instead of the strongest baseline neurons for image editing | GILL image editing · boy → girl | Perturbation ℓ2 norm: 0.390 -> 0.350 | Helped |
| Use NAM-identified I-neurons instead of the strongest baseline neurons for image editing | GILL image editing · dog → cat | Perturbation ℓ2 norm: 0.259 -> 0.255 | Helped |
| Use NAM-identified I-neurons instead of the strongest baseline neurons for image editing | GILL image editing · cantaloupe → apple | Perturbation ℓ2 norm: 0.324 -> 0.316 | Helped |
Surprising findings
- T-neurons and I-neurons for the same semantics only partially overlap, indicating modality-specific neurons even for the same concept.
Limitations
- Experiments limited to two MLLMs (GILL and NExT-GPT) and text/image outputs only.
- Relies on external segmentation (EVA02) and attribution (Diffusers-Interpret) tools.
- Indirect-influence term is a heuristic rather than a principled derivation.
Experimental setup
Datasets: COCO
Benchmarks: COCO
Metrics: CLIPScore BERTScore MoverScore BLEURT cross-sample invariance ratio semantic specificity ratio l2 perturbation norm for editing
Baselines: AcT AcU GraD GraT GraI CE
Builds on: knowledge neurons skill neurons multi-modal neurons Diffusers-Interpret causal tracing
Closest papers by meaning
- Cross-Modal Unlearning via Influential Neuron Path Editing in Multimodal Large Language Models (AAAI 2026 · similarity 0.91)
- Identifying Query-Relevant Neurons in Large Language Models for Long-Form Texts (AAAI 2025 · similarity 0.88)
- What does the Knowledge Neuron Thesis Have to do with Knowledge? (ICLR 2024 · similarity 0.88)
- Do LLMs Signal When They’re Right? Evidence from Neuron Agreement (ICML 2026 · similarity 0.87)
- Precise Localization of Memories: A Fine-grained Neuron-level Knowledge Editing Technique for LLMs (ICLR 2025 · similarity 0.87)
- Edit Less, Achieve More: Dynamic Sparse Neuron Masking for Lifelong Knowledge Editing in LLMs (NeurIPS 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.