Research map / Architectures and efficiency / Attention and transformers
On the Faithfulness of Vision Transformer Explanations
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SaCo, a pairwise salience-guided faithfulness metric for Vision Transformer explanations, showing existing cumulative-perturbation metrics cannot separate advanced explanation methods from Random Attribution.
Problem
Existing perturbation-based faithfulness metrics for Vision Transformer post-hoc explanations use cumulative pixel removal and ignore salience magnitudes, so they fail to test whether salience differences reflect differences in pixel influence and cannot distinguish good explanations from random one
Approach
Partition pixels into K salience-ranked subsets, perturb each subset individually (replace with per-sample mean), and compute a Kendall-tau-like coefficient that rewards pairs whose influence order matches salience order, weighted by the salience difference; random attribution scores near zero by construction.
Key results
- SaCo scores Random Attribution near zero (e.g., 0.0019 on CIFAR-10) while existing metrics like AOPC and AUC rank Random Attribution comparably to advanced methods.
- Correlation between SaCo sample rankings and existing metrics ranges 0.18-0.22, versus 0.48 average among existing metrics.
- Adding gradient information to attention raises SaCo from 0.1835 to 0.3783 (last layer) and multi-layer aggregation plus gradient reaches 0.4558 on ImageNet.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add gradient information to final-layer attention (ablation) | Attention-based explanation · ImageNet | SaCo: 0.1835 -> 0.3783 | Helped |
| Add cross-layer aggregation to attention without gradient (ablation) | Attention-based explanation · ImageNet | SaCo: 0.1835 -> 0.2453 | Helped |
| Add cross-layer aggregation when using gradient information (ablation) | Attention-based explanation · ImageNet | SaCo: 0.3783 -> 0.4558 | Helped |
| Increase the number of pixel subsets from K=5 to K=20 (ablation) | Most explanation methods · ImageNet | SaCo: Partial LRP: 0.4628 -> 0.2282; Transformer Attribution: 0.5680 -> 0.2906; Transformer-MM: 0.6041 -> 0.3651 | Hurt |
| Increase the number of pixel subsets from K=5 to K=20 (ablation) | ATTCAT · ImageNet | SaCo: 0.3178 -> 0.3617 | Mixed |
Surprising findings
- Random Attribution scores comparably to or above some state-of-the-art methods on AUC, AOPC, and LOdds metrics.
- Integrated Gradients performs best on AOPC, LOdds, and AUC but worst on Comprehensiveness, showing removal-order inconsistency of existing metrics.
Limitations
- SaCo score depends on the number of subsets K, with most methods declining as K increases.
- Perturbation with per-sample mean may cause out-of-distribution effects, as noted by the authors' discussion of confidence drops.
Experimental setup
Datasets: CIFAR-10 CIFAR-100 ImageNet
Benchmarks: CIFAR-10 CIFAR-100 ImageNet
Metrics: SaCo AUC AOPC LOdds Comprehensiveness
Baselines: AUC AOPC LOdds Comprehensiveness Random Attribution
Builds on: Kendall tau statistic Transformer Attribution AOPC Rollout
Closest papers by meaning
- Probabilistic Conceptual Explainers: Trustworthy Conceptual Explanations for Vision Foundation Models (ICML 2024 · similarity 0.89)
- Improving Interpretation Faithfulness for Vision Transformers (ICML 2024 · similarity 0.89)
- Token Transformation Matters: Towards Faithful Post-hoc Explanation for Vision Transformer (CVPR 2024 · similarity 0.89)
- Saliency-R1: Enforcing Interpretable and Faithful Vision-language Reasoning via Saliency-map Alignment Reward (CVPR 2026 · similarity 0.87)
- LibraGrad: Balancing Gradient Flow for Universally Better Vision Transformer Attributions (CVPR 2025 · similarity 0.86)
- Measuring Cross-Modal Synergy: A Benchmark for VLM Explainability (NeurIPS 2026 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.