Research map / Architectures and efficiency / Attention and transformers
Analyzing Vision Transformers for Image Classification in Class Embedding Space
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| zero attention weights between image tokens and all other image tokens (ablation) | ViT-B/32 image tokens · ImageNet-S validation | class identifiability rate (%): 60.73% to 0.12% | Hurt |
| remove attention weights between image tokens and the [CLS] token (ablation) | ViT-B/32 image tokens · ImageNet-S validation | class identifiability rate | No clear effect |
| remove class-labeled tokens, leaving context-labeled tokens (ablation) | ViT-B/32 · ImageNet-S validation | class identifiability rate of context-labeled tokens: 56.24% to 38.68% | Hurt |
| remove context-labeled tokens, leaving class-labeled tokens (ablation) | ViT-B/32 · ImageNet-S validation | class identifiability rate of class-labeled tokens: 71.91% to 44.70% | Hurt |
| use framework-derived importance to rank tokens, then remove least-to-most important | ViT-B/32 explainability framework · ViT-B/32 token-removal perturbation | AUC of accuracy scores: 84.54 % (Chefer et al.) vs 83.63% (Ours) | Mixed |
| use framework-derived importance to rank tokens, then remove most-to-least important | ViT-B/32 explainability framework · ViT-B/32 token-removal perturbation | AUC of accuracy scores: 39.29% (Chefer et al.) vs 41.28 % (Ours) | Mixed |
| remove up to 60% of the least-important tokens (ablation) | ViT-B/32 · ViT-B/32 token-removal perturbation | classification accuracy | No clear effect |
| use linear probes instead of class-embedding projection to score token relevance | ViT-B/32 · ImageNet-S token perturbation | prediction of token relevance to the categorical decision | Hurt |
| add class-embedding projection of intermediate token states rather than train a linear probe | ViT-B/32 · ImageNet-S token perturbation | prediction of token relevance to the categorical decision | Helped |
Closest papers by meaning
- A Simple Interpretable Transformer for Fine-Grained Image Classification and Analysis (ICLR 2024 · similarity 0.89)
- Revisiting [CLS] and Patch Token Interaction in Vision Transformers (ICLR 2026 · similarity 0.88)
- Vision Transformers Need Registers (ICLR 2024 · similarity 0.87)
- Rethinking Decoders for Transformer-based Semantic Segmentation: A Compression Perspective (NeurIPS 2024 · similarity 0.86)
- Beyond the Doors of Perception: Vision Transformers Represent Relations Between Objects (NeurIPS 2024 · similarity 0.86)
- ConceptAttention: Diffusion Transformers Learn Highly Interpretable Features (ICML 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.