Research map / Vision-language and multimodal / Human and embodied modeling

Head Pursuit: Probing Attention Specialization in Multimodal Transformers

NeurIPS 2025 · Vision-language and multimodal › Human and embodied modeling · approach: Language-grounded visual concept learning · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Applies sparse-coding (SOMP) over the unembedding matrix to rank attention heads by how well they specialize in target concepts, then inverts or amplifies the top ~1% of heads to suppress or enhance those concepts in text and vision-language outputs.

Problem

Attention heads in generative language and vision-language models are poorly understood at the component level, and existing logit-lens style probes work per-sample and are heuristic, making it hard to identify which heads drive a given concept.

Approach

Reinterpret the Logit Lens as a single step of Matching Pursuit and extend it to Simultaneous OMP over many samples, using a concept-restricted unembedding matrix as dictionary; the variance fraction explained per head serves as a specialization score for head selection and intervention, with no extra training.

Key results

• Inverting 32 SOMP-selected heads in Mistral-7B reduces toxic generations on RTP to 0.66 of baseline vs 1.13 for random heads.
• On LLaVA-NeXT-7B, inverting 32 SOMP-selected heads drives MNIST classification accuracy to 0.12 of base, while random heads at the same layers keep it at 0.99.
• On Flickr30k captioning with LLaVA-NeXT-7B, inhibiting 16 color heads reduces color-keyword count to 0.18 with CIDEr at 0.91 of original; enhancing with alpha=5 raises sentiment-keyword count to about 5x.

Measured effects reported in the paper

Surprising findings

• Logit Lens-selected heads are relevant for task performance but degrade in-domain and out-of-domain examples equally, so they lack specificity.
• Heads selected on one dataset transfer to semantically related datasets, e.g., digit-related heads from MNIST/SVHN also degrade GTSRB traffic-sign recognition.

Limitations

• SOMP assumes linear structure that may not capture nonlinear head representations.
• Results depend on the quality and coverage of the keyword dictionary; incomplete lists can bias head selection.
• Interventions are global rescalings or inversions of head outputs, which are coarse compared with finer-grained steering.

Experimental setup

Datasets: TriviaQA RealToxicityPrompts Thoroughly Engineered Toxicity MNIST SVHN GTSRB EuroSAT RESISC45 DTD Flickr30k

Benchmarks: TriviaQA RealToxicityPrompts Thoroughly Engineered Toxicity MNIST SVHN GTSRB EuroSAT RESISC45 DTD Flickr30k

Metrics: F1 normalized toxic generation count toxic keyword frequency exact-match classification accuracy property keyword count CIDEr

Baselines: Logit Lens head selection random head selection

Builds on: Logit Lens Matching Pursuit Orthogonal Matching Pursuit Attention Lens Tuned Lens

Closest papers by meaning

• Unveiling Visual Perception in Language Models: An Attention Head Analysis Approach (CVPR 2025 · similarity 0.92)
• Where Does Vision Meet Language? Understanding and Refining Visual Fusion in MLLMs via Contrastive Attention (CVPR 2026 · similarity 0.90)
• Cross-Attention Head Position Patterns Can Align with Human Visual Concepts in Text-to-Image Generative Models (ICLR 2025 · similarity 0.89)
• See What You Are Told: Visual Attention Sink in Large Multimodal Models (ICLR 2025 · similarity 0.89)
• Talking Heads: Understanding Inter-Layer Communication in Transformer Language Models (NeurIPS 2024 · similarity 0.88)
• Seeing is Understanding: Unlocking Causal Attention into Modality-Mutual Attention for Multimodal LLMs (ICML 2026 · similarity 0.88)