AI Paper Atlas

Research map / Architectures and efficiency / Attention and transformers

From Weights to Concepts: Data-Free Interpretability of CLIP via Singular Vector Decomposition

Francesco Gentile, Nicola Dall'Asen, Francesco Tonini, Massimiliano Mancini, Lorenzo Vaquero, Elisa Ricci

CVPR 2026 · Architectures and efficiency › Attention and transformers · approach: Token merging and masked pretraining · 1 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
suppress background-related singular vectors found by SITHOpenCLIP ViT-L/14 · Waterbirdsworst-group accuracy (%): 47.9 -> 70.6Improved
suppress singular vectors instead of zero-ablating whole heads (TextSpan) (ablation)OpenCLIP ViT-L/14 · Waterbirdsworst-group accuracy (%): 68.0 -> 70.6Improved
randomly ablate same number of singular values (ablation)OpenCLIP ViT-L/14 · Waterbirdsoverall accuracy (%): 73.5 -> 72.0Hurt
suppress NSFW singular vectorsOpenCLIP ViT-L/14 · ViSU unsafe image queryR@10: 38.9 -> 40.4Improved
suppress NSFW singular vectorsOpenCLIP ViT-L/14 · ViSU safe text queryR@10: 75.1 -> 74.5No clear effect
rescale singular values by similarity to class namesOpenCLIP ViT-L/14 · Flowers 102 / FGVC-Aircraft / DTDtop-1 accuracy (%): 76.5 -> 77.5 (Flowers 102)Improved
randomly choose singular-value scaling factors (ablation)OpenCLIP ViT-L/14 · Flowers 102 / FGVC-Aircraft / DTDtop-1 accuracy (%): 76.5 -> 76.4 (Flowers 102)Hurt
add coherence term (COMP) to NNOMP concept selection (ablation)SITH · last layer singular vectorsinterpretability vs fidelityImproved
replace singular vectors with SITH reconstructions (ablation)OpenCLIP ViT-L/14 layer 23 · ImageNet, Flowers 102, Oxford Pets, Stanford Carszero-shot accuracyNo clear effect

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy