Research map / Vision-language and multimodal / Multimodal LLMs

LatentLens: Revealing Highly Interpretable Visual Tokens in LLMs

ICML 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Training-free token reduction for MLLMs · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes LatentLens, a training-free lens that maps visual token representations in VLMs to nearest contextualized text representations from a Visual Genome corpus, finding most visual tokens are interpretable across layers unlike with LogitLens or EmbeddingLens.

Problem

Existing training-free lenses (LogitLens, EmbeddingLens) suggest visual tokens in LLMs are rarely interpretable, leaving unclear what visual token representations encode at each LLM layer and why frozen LLMs can process them so easily.

Approach

Compare latent visual token representations to a large pool of contextualized token representations from sentences (encoded by the same LLM across layers), rather than to the input embedding or unembedding matrix, and return the top-k matching sentences as descriptions.

Key results

• Averaged across 15 VLM configurations and layers, LatentLens renders 68% of visual tokens interpretable versus 32% for EmbeddingLens and 24% for LogitLens.
• LatentLens achieves 86% (Molmo-7B-D) and 78% (Molmo-72B) average interpretability on off-the-shelf models, outperforming baselines on all six.
• LLM judge agrees with human annotators with Cohen's kappa = 0.68 (accuracy 84.4%) over 1,020 annotations.

Surprising findings

• Visual tokens at the input and early layers align most strongly to mid-layer contextual representations (e.g., layers 8-16), the 'Mid-Layer Leap'.
• Replacing captioning with a spatial yes/no-style task drops interpretability by about 30% and yields zero overlap with the original nearest neighbors.

Limitations

• Requires pre-computing and storing a large contextual embedding corpus (about 26 GB for 8 layers in float8).
• Interpretability judgments depend on the Visual Genome corpus and an LLM judge (kappa 0.68), which may be subjective.
• Effects on downstream VLM behavior are not studied; larger models show less consistent trends across layers.

Experimental setup

Datasets: PixMo-Cap Visual Genome PixMoCap

Benchmarks: PixMo-Cap validation set (interpretability) DCScore

Metrics: % interpretable visual tokens Cohen's kappa DCScore top-5 nearest neighbor overlap

Baselines: LogitLens EmbeddingLens Tuned Lens

Builds on: LogitLens EmbeddingLens Tuned Lens Molmo

Closest papers by meaning

• What Do Visual Tokens Really Encode? Uncovering Sparsity and Redundancy in Multimodal Large Language Models (CVPR 2026 · similarity 0.91)
• Rethinking Visual Token Reduction in LVLMs Under Cross-Modal Misalignment (AAAI 2026 · similarity 0.90)
• Towards Interpreting Visual Information Processing in Vision-Language Models (ICLR 2025 · similarity 0.90)
• Unified Language-Vision Pretraining in LLM with Dynamic Discrete Visual Tokenization (ICLR 2024 · similarity 0.90)
• Learning to See Before Seeing: Demystifying LLM Visual Priors from Language Pre-training (ICLR 2026 · similarity 0.89)
• Fill the GAP: A Granular Alignment Paradigm for Visual Reasoning in Multimodal Large Language Models (NeurIPS 2026 · similarity 0.89)