Research map / Architectures and efficiency / Attention and transformers
Explicitly Modeled Attention Maps for Image Classification
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace key-query attention maps with Gaussian ExpAtt maps | ResNet50 · CIFAR-10 | accuracy: 90.78 -> 90.99 | Helped |
| replace key-query attention maps with Gaussian ExpAtt maps | ResNet50 · CIFAR-100 | accuracy: 80.32 -> 81.02 | Helped |
| use Gaussian ExpAtt instead of AA attention | ResNet50 · ImageNet | Top1 accuracy: 77.70 -> 78.13 | Helped |
| use Gaussian ExpAtt instead of AA attention | ResNet101 · ImageNet | Top1 accuracy: 78.70 -> 79.56 | Helped |
| use Gaussian ExpAtt instead of AA attention | ResNet152 · ImageNet | Top1 accuracy: 79.10 -> 80.02 | Helped |
| use Gaussian ExpAtt instead of AA attention | ResNet34 · ImageNet | Top1 accuracy: 74.70 -> 74.24 | Hurt |
| replace AA attention with Gaussian ExpAtt | MobileNetV2 · Tiny ImageNet | accuracy: 65.89 -> 66.14 | Helped |
| share Gaussian attention maps across heads instead of using different maps (ablation) | ResNet50 · CIFAR-100 | Top1 accuracy: 80.78 -> 81.02 | Helped |
| combine content-independent ExpAtt with content-based attention (ablation) | ResNet50 · CIFAR-100 | accuracy: 79.46 -> 78.19 | Hurt |
| add the element-wise plus-one offset to Gaussian attention maps to retain global features (ablation) | ResNet50 · CIFAR-100 | Top1 accuracy: 80.42 -> 81.02 | Helped |
Closest papers by meaning
- Exploring Self-Attention for Image Recognition (CVPR 2020 · similarity 0.91)
- Stand-Alone Self-Attention in Vision Models (NeurIPS 2019 · similarity 0.90)
- Scaling Local Self-Attention for Parameter Efficient Visual Backbones (CVPR 2021 · similarity 0.89)
- Unveiling the Hidden Structure of Self-Attention via Kernel Principal Component Analysis (NeurIPS 2024 · similarity 0.87)
- Bridging the Divide: Reconsidering Softmax and Linear Attention (NeurIPS 2024 · similarity 0.87)
- Revisiting the Integration of Convolution and Attention for Vision Backbone (NeurIPS 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.