Research map / Architectures and efficiency / Attention and transformers
Robustifying Token Attention for Vision Transformers
Yong Guo, David Stutz, Bernt Schiele
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add TAP and ADL together to FAN-B-Hybrid | FAN-B-Hybrid · ImageNet-C | mCE: 46.1 -> 43.7 | Improved |
| use dilated average pooling in TAP instead of multi-kernel pooling (ablation) | TAP on FAN-B-Hybrid · ImageNet-C | mCE: 45.5 -> 44.9 | Improved |
| replace adaptive TAP with fixed AvgPool5x5 (ablation) | FAN-B-Hybrid · ImageNet | top-1 accuracy: 83.9 -> 82.7 | Hurt |
| add TAP in every block rather than only shallow or middle+deep layers (ablation) | FAN-B-Hybrid · ImageNet-C | mCE: 45.7 -> 44.9 | Improved |
| increase ADL weight lambda to 10 (ablation) | ADL · ImageNet-C | mCE | Hurt |
| set ADL threshold tau=2 rather than 5 (ablation) | ADL on FAN-B-Hybrid · ImageNet-C | mCE: 45.8 -> 44.4 | Improved |
| increase TAP branches K beyond 4 (ablation) | TAP · ImageNet-C | mCE | No clear effect |
| add ADL alone to FAN-B-Hybrid (ablation) | FAN-B-Hybrid · ImageNet | top-1 accuracy: 83.9 -> 84.0 | No clear effect |
| add TAP and ADL to Swin-B | Swin-B · ImageNet-C | mCE: 54.4 -> 51.9 | Improved |
| add TAP and ADL to FAN-B-Hybrid for segmentation | FAN-B-Hybrid · ACDC test | mIoU: 60.6 -> 63.6 | Improved |
| add ADL to reduce attention similarity between heads | FAN-B-Hybrid · ImageNet head similarity | cosine similarity between heads: 0.63 -> 0.27 | Improved |
Closest papers by meaning
- Understanding The Robustness in Vision Transformers (ICML 2022 · similarity 0.90)
- DynamicViT: Efficient Vision Transformers with Dynamic Token Sparsification (NeurIPS 2021 · similarity 0.89)
- Saliency-Driven Token Merging for Vision Transformers (CVPR 2026 · similarity 0.89)
- Improving Adversarial Transferability on Vision Transformers via Forward Propagation Refinement (CVPR 2025 · similarity 0.88)
- Vision Transformers Don't Need Trained Registers (NeurIPS 2025 · similarity 0.88)
- ImagePiece: Content-aware Re-tokenization for Efficient Image Recognition (AAAI 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
