Research map / Vision-language and multimodal / Cross-modal learning
SAE-V: Interpreting Multimodal Models for Enhanced Alignment
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Extends sparse autoencoders to multimodal LLMs (SAE-V) to interpret features and uses a cross-modal cosine-similarity score over SAE features to filter alignment data, reporting gains with less data.
Problem
Interpretability of multimodal LLMs is harder than for text-only LLMs because of image-text fusion, and existing interpretability work rarely feeds back into alignment or data quality; SAE tools have not been adapted to MLLMs.
Approach
Train sparse autoencoders on MLLM hidden states (SAE-V), weight each feature by the cosine similarity between its top text-token and vision-token activations, and score training samples by summing the weights of their activated features to rank and filter data for DPO alignment.
Key results
- SAE-V-based filtering with 50% of Align-Anything (co-occurrence filter) reached 108.17 on LLaVA-Bench, 115% of full-dataset performance, while random selection stayed near baseline.
- On Chameleon-7B, the cosine filter reached 52.13 (120% of full-data performance) with 70% data; on RLAIF-V it reached 125% with 10% of data.
- SAE-V reconstruction loss was 38.3% and 50.6% of the standard SAE's on MLLMs and LLMs respectively, per the authors' claim.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Apply SAE-V cosine-similarity filtering to select 20% of Align-Anything data | LLaVA-NeXT-7B · Align-Anything (1/20 subset) | performance: 94.2 -> 106.8 | Helped |
| Apply SAE-V cosine-similarity filtering to select 30% of Align-Anything data | LLaVA-NeXT-7B · Align-Anything (1/20 subset) | performance: 94.2 -> 114.9 | Helped |
| Use SAE-V cosine-similarity ranking instead of random selection at 20% data (ablation) | LLaVA-NeXT-7B · Align-Anything (1/20 subset) | performance: 106.8 vs 97.0 | Helped |
| Use SAE-V cosine-similarity ranking instead of random selection at 30% data (ablation) | LLaVA-NeXT-7B · Align-Anything (1/20 subset) | performance: 114.9 vs 98.3 | Helped |
| Use SAE-V filtering with 40% of the data instead of the full dataset | LLaVA-NeXT-Vicuna-13B · Align-Anything | performance: 111.2 -> 116.7 | Helped |
| Use SAE-V filtering instead of random selection at 40% data (ablation) | LLaVA-NeXT-Vicuna-13B · Align-Anything | performance: 116.7 vs 105.8 | Helped |
| Replace SAE with SAE-V feature extraction | Chameleon-7B · Chameleon-7B | L0: 24757.6 -> 50.1 | Helped |
| Replace SAE with SAE-V feature extraction | Anole-7B · Anole-7B | L0: 62.1 -> 50.1 | Helped |
| Use SAE-V filtering rather than random filtering on larger data proportions (ablation) | Chameleon-7B · Align-Anything | performance: 52.13 (120% of the performance on full dataset) with 70% data | Helped |
| Use cosine-similarity filtering rather than the full RLAIF-V dataset (ablation) | Chameleon-7B · RLAIF-V | performance: 54.23 (125% of the full dataset's performance) using only 10% of the data | Helped |
| Use cosine-similarity filtering rather than the full MMInstruct dataset (ablation) | Chameleon-7B · MMInstruct | performance: 61.2 (117% of the full dataset's performance) with 60% data | Helped |
| Replace cosine-similarity data ranking with random selection (ablation) | LLaVA-NeXT-7B · Align-Anything | performance: At any data filter proportion, the SAE-V-based data filtering method outperforms the random selection baseline. | Hurt |
Surprising findings
- For LLaVA-NeXT models, SAE-V required more activated features (higher L0) than a standard SAE, contrary to the Chameleon and Anole results.
- Filtering to a moderate data fraction often outperformed the full dataset, and performance declined as more data was included.
Limitations
- Theoretical basis for the cross-modal similarity metric and its link to performance is not established.
- Evaluated only on text and vision modalities; audio, video, and embodied settings untested.
- Alignment results rely on a single benchmark family (LLaVA-Bench) and small filtered subsets, with variance across ratios.
Experimental setup
Datasets: The Pile OBELICS Align-Anything ImageNet RLAIF-V MMInstruct A-OKVQA
Benchmarks: LLaVA-Bench ImageNet A-OKVQA
Metrics: L0 sparsity reconstruction loss LLaVA-Bench score classification accuracy VQA accuracy
Baselines: random selection IFD L0 ranking co-occurrence L0 ranking
Builds on: sparse autoencoders direct preference optimization IFD
Closest papers by meaning
- Strong Helps Weak: Directional Cross-Modal Alignment Transfer in Multi-modal LLMs (NeurIPS 2026 · similarity 0.92)
- Implicit Multimodal Alignment: On the Generalization of Frozen LLMs to Multimodal Inputs (NeurIPS 2024 · similarity 0.90)
- DeepAlign: Mitigating Modality Conflict through Modality-Specific Alignment (CVPR 2026 · similarity 0.90)
- ModaVerse: Efficiently Transforming Modalities with LLMs (CVPR 2024 · similarity 0.90)
- LoMo: Local Modality Substitution for Deeper Vision-Language Fusion (NeurIPS 2026 · similarity 0.90)
- AlignVLM: Bridging Vision and Language Latent Spaces for Multimodal Document Understanding (NeurIPS 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.