Research map / Architectures and efficiency / Attention and transformers
Token Transformation Matters: Towards Faithful Post-hoc Explanation for Vision Transformer
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes TokenTM, a post-hoc explanation method for Vision Transformers that combines attention weights with a token transformation measure (relative length and cosine direction correlation) aggregated across all layers.
Problem
Existing post-hoc explanation methods for Vision Transformers use only attention weights and ignore how transformations of tokens change their contributions, leading to unfaithful explanations.
Approach
Reinterpret MHSA and FFN outputs as weighted sums of original and transformed tokens, measure token transformation via relative L2 length change and normalized exponential cosine correlation, and aggregate these weights with attention across layers via initialization and layer-wise update rules.
Key results
- On ImageNet-Segmentation with DeiT, TokenTM reaches 81.79% pixel accuracy and 64.22% mIoU, versus 79.05% and 61.56% for GAE.
- On CIFAR-10 perturbation tests (predicted label), TokenTM achieves negative AUC 74.90 and positive AUC 37.14, versus 71.67 and 37.77 for GAE.
- Ablation shows aggregating more layers consistently improves segmentation (mIoU 42.99 at 3 layers to 60.29 at 12 layers on ImageNet with ViT-B).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add TokenTM's token-transformation measurement and aggregation framework | DeiT explanation method · ImageNet-Seg | mIoU: 61.56 -> 64.22 | Helped |
| add TokenTM's token-transformation measurement and aggregation framework | DeiT explanation method · CIFAR-10, predicted label | AUC negative perturbation: 71.67 -> 74.90 | Helped |
| add TokenTM's token-transformation measurement and aggregation framework | DeiT explanation method · CIFAR-100, predicted label | AUC positive perturbation: 17.52 -> 16.07 | Helped |
| add TokenTM's token-transformation measurement and aggregation framework | DeiT explanation method · ImageNet, predicted label | AUC negative perturbation: 57.53 -> 58.12 | Helped |
| add TokenTM's token-transformation measurement and aggregation framework | ViT-B · Impact on probability | AOPC: 0.721 -> 0.764 | Helped |
| add aggregation framework (AF) (ablation) | Baseline explanation method on ViT-B · ImageNet | Pixel accuracy: 52.94 -> 76.30 | Helped |
| add relative length measurement to AF (ablation) | Baseline + AF on ViT-B · ImageNet | Pixel accuracy: 76.30 -> 77.50 | Helped |
| add NECC to AF and relative length measurement (ablation) | Baseline + AF + L on ViT-B · ImageNet | Pixel accuracy: 77.50 -> 78.22 | Helped |
| aggregate more Transformer layers (ablation) | TokenTM on ViT-B · ImageNet | Pixel accuracy: 63.63 -> 68.38 -> 70.46 -> 78.22 | Helped |
| aggregate more Transformer layers (ablation) | TokenTM on ViT-B · ImageNet | AUC positive perturbation: 22.57 -> 20.17 -> 19.24 -> 16.18 | Helped |
Surprising findings
- Attention-only methods such as Raw Attention and ATTCAT can misidentify foreground regions, with ATTCAT scoring 43.88% pixel accuracy on ImageNet-Segmentation.
Limitations
- Evaluation limited to DeiT and ViT variants on classification and segmentation-style localization.
- Relies on hand-designed length and cosine measures; the choice of L2 norm and NECC is not compared against alternatives in depth.
Experimental setup
Datasets: ImageNet-Segmentation CIFAR-10 CIFAR-100 ImageNet
Benchmarks: ImageNet-Segmentation CIFAR-10 CIFAR-100 ImageNet
Metrics: Pixel accuracy mIoU mAP Negative/positive perturbation AUC AOPC LOdds
Baselines: Grad-CAM LRP Conservative LRP Transformer Attribution Raw Attention Rollout ATTCAT GAE
Builds on: Transformer attention rollout Generic Attention Explainability (GAE) Transformer Attribution Normalized Exponential Cosine Correlation
Closest papers by meaning
- On the Faithfulness of Vision Transformer Explanations (CVPR 2024 · similarity 0.89)
- TransPrune: Token Transition Pruning for Efficient Large Vision-Language Model (CVPR 2026 · similarity 0.87)
- LearnPruner: Rethinking Attention-based Token Pruning in Vision Language Models (ICLR 2026 · similarity 0.87)
- Probabilistic Conceptual Explainers: Trustworthy Conceptual Explanations for Vision Foundation Models (ICML 2024 · similarity 0.87)
- Register and [CLS] tokens induce a decoupling of local and global features in large ViTs (NeurIPS 2025 · similarity 0.87)
- Interpreting vision transformers via residual replacement model (NeurIPS 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.