Research map / Vision-language and multimodal / Multimodal LLMs
Sparse Autoencoders Learn Monosemantic Features in Vision-Language Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Extends sparse autoencoders to vision-language models (CLIP, SigLIP, AIMv2, WebSSL), proposes a monosemanticity score validated by a 1000-question user study, and shows SAE neuron interventions on CLIP steer LLaVA outputs.
Problem
Neurons in vision-language model layers are polysemantic, and there is no neuron-level metric validated against human judgment to evaluate how monosemantic SAE latents are, which limits their use for interpreting and steering VLMs.
Approach
A MonoSemanticity score (MS) that averages pairwise embedding cosine similarities of images weighted by pairwise min-max-normalized neuron activations, validated by a large Mechanical Turk study; SAE latents (especially Matryoshka BatchTopK with wide, sparse settings) are shown to be more monosemantic than raw neurons, and a single SAE neuron on CLIP's vision encoder can be intervened on to steer
Key results
- Human study with 1000 questions and 71 annotators: MS agreed with majority human choice in 82.8% of pairs, rising from 56.6% (MS difference 0.0-0.1) to 100% (0.8-0.9).
- Best neuron MS for CLIP ViT-L last layer rises from 0.01 (no SAE) to 0.82 (Matryoshka, x1) and 1.00 (x64), with expansion factor 64 reaching 1.00 at all layers.
- For LLaVA steering, SAE directions achieved 52.5% both-criteria concept suppression vs 33.3% for DiffMean, and 42.4% vs 35.8% for concept insertion.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Apply an SAE to CLIP ViT-L layer 22 rather than using the original neurons | CLIP ViT-L layer 22 · Highest-scoring neuron | MS: 0.01 -> 0.80 | Helped |
| Use Matryoshka BatchTopK rather than BatchTopK at expansion ×4 (ablation) | CLIP ViT-L layer 22 SAE · Highest-scoring neuron | MS: 0.80 -> 0.87 | Helped |
| Increase BatchTopK expansion from ×1 to ×4 (ablation) | CLIP ViT-L layer 22 SAE · All neurons | average MS: 0.14 ± 0.12 -> 0.20 ± 0.17 | Helped |
| Increase Matryoshka expansion from ×1 to ×4 (ablation) | CLIP ViT-L layer 22 SAE · All neurons | average MS: 0.17 ± 0.17 -> 0.23 ± 0.19 | Helped |
| Increase the expansion factor beyond ×4 (ablation) | Matryoshka SAE neurons · CLIP ViT-L last layer | MS across neurons | Mixed |
| Increase sparsity by lowering K (ablation) | Matryoshka SAE · Neuron monosemanticity | MS | Helped |
| Lower K to 1 rather than use K=50 (ablation) | Matryoshka SAE · Reconstruction | R²: 74.9% -> 31.3% | Hurt |
| Use Matryoshka reconstruction objective rather than standard BatchTopK (ablation) | CLIP ViT-L SAE · Same expansion factor | R²: 2 or 3 points lower R² | Hurt |
| Steer with SAE directions rather than DiffMean | LLaVA-1.5-7B · Concept insertion | both criteria satisfied (%): 35.8 -> 42.4 | Helped |
| Steer with SAE directions rather than DiffMean | LLaVA-1.5-7B · Concept suppression | unrelated concept kept (%): 38.7 -> 81.4 | Helped |
| Steer with SAE directions rather than DiffMean | LLaVA-1.5-7B · Concept suppression | desired concept removed (%): 64.0 -> 64.4 | No clear effect |
| Clamp a selected SAE neuron to α=100 | LLaVA-1.5-7B · White images | image-text similarity: 0.212 ± 0.021 -> 0.259 ± 0.036 | Helped |
Surprising findings
- About 90% of neurons in a same-width (expansion factor 1) SAE outperform the original neurons on MS.
- MS of the best neurons keeps increasing with expansion factor, but the average-neuron score peaks around expansion factor 4 and then decreases for very wide latents.
Limitations
- High-MS neurons do not always give precise steering effects; e.g., a golden retriever neuron can trigger any dog-related output.
- A fraction of SAE feature-detector neurons show no clear steering effect.
- MS is only evaluated on vision representations and the approach is focused on image concepts; the benchmark and study use ImageNet images.
Experimental setup
Datasets: ImageNet iNaturalist
Benchmarks: MS user-study benchmark (1000 neuron pairs) ImageNet validation activations
Metrics: MonoSemanticity score alignment rate with human judgment R^2 reconstruction L0 sparsity concept insertion/suppression success rate
Baselines: No SAE (raw neurons) Difference-in-Means steering BatchTopK SAE
Builds on: Sparse autoencoders (ReLU/L1) BatchTopK SAE Matryoshka SAE CLIP LLaVA
Closest papers by meaning
- Interpreting CLIP with Hierarchical Sparse Autoencoders (ICML 2025 · similarity 0.91)
- SAEBench: A Comprehensive Benchmark for Sparse Autoencoders in Language Model Interpretability (ICML 2025 · similarity 0.91)
- Sparse Autoencoders, Again? (ICML 2025 · similarity 0.90)
- Rethinking Evaluation of Sparse Autoencoders through the Representation of Polysemous Words (ICLR 2025 · similarity 0.90)
- Cascaded Sparse Autoencoders LearnMulti-Level Visual Concepts in Multimodal LLMs (NeurIPS 2026 · similarity 0.90)
- Automatically Interpreting Millions of Features in Large Language Models (ICML 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.