Research map / Architectures and efficiency / Attention and transformers
Hard-Attention for Scalable Image Classification
Athanasios Papadopoulos, Pawel Korus, Nasir Memon
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use TNet hard attention instead of the fully convolutional BagNet-77 baseline | ImageNet classification · ImageNet ILSVRC 2012 validation | Top-1 accuracy: 73.42 % -> 74.12 % | Improved |
| Use TNet with 1 attended location instead of Saccader with 30 locations | ImageNet classification · ImageNet ILSVRC 2012 validation | Top-1 accuracy: 72.27 % -> 73.12 % | Improved |
| Use TNet with 1 attended location instead of BagNet-77 | ImageNet classification · ImageNet ILSVRC 2012 validation | FLOPs: 18.42 -> 3.63 | Improved |
| Use TNet hard attention at 448² rather than resizing the whole image for EfficientNet-B0 | fMoW classification · fMoW | Top-1 accuracy: 69.83 % -> 71.46 % | Improved |
| Use TNet hard attention at 448² rather than resizing the whole image for EfficientNet-B0 | fMoW classification · fMoW | FLOPs: 1.54 -> 1.16 | Improved |
| Use TNet with 4 locations at 896² rather than resizing the whole image for EfficientNet-B0 | fMoW classification · fMoW | Top-1 accuracy: 70.6 % -> 72.16 % | Improved |
| Use TNet with 4 locations at 896² rather than resizing the whole image for EfficientNet-B0 | fMoW classification · fMoW | FLOPs: 6.18 -> 1.94 | Improved |
| Add per-feature regularization (ablation) | TNet · ImageNet ILSVRC 2012 validation, 1 location | Top-1 accuracy: 65.21 % -> 73.12 % | Improved |
| Add per-feature regularization (ablation) | TNet · ImageNet ILSVRC 2012 validation, 3 locations | Top-1 accuracy: 67.33 % -> 74.41 % | Improved |
| Add per-feature regularization (ablation) | TNet · ImageNet ILSVRC 2012 validation, 5 locations | Top-1 accuracy: 68.55 % -> 74.62 % | Improved |
| Include the global contextual feature alongside attended-location features (ablation) | TNet · ImageNet ILSVRC 2012 validation, 2 locations | Top-1 accuracy: 67.95 % -> 74.12 % | Improved |
| Include the global contextual feature alongside attended-location features (ablation) | TNet · ImageNet ILSVRC 2012 validation, 3 locations | Top-1 accuracy: 69.66 % -> 74.41 % | Improved |
Closest papers by meaning
- ImageNet-Hard: The Hardest Images Remaining from a Study of the Power of Zoom and Spatial Biases in Image Classification (NeurIPS 2023 · similarity 0.87)
- Glance and Focus: a Dynamic Approach to Reducing Spatial Redundancy in Image Classification (NeurIPS 2020 · similarity 0.85)
- Multi-Scale Dense Networks for Resource Efficient Image Classification (ICLR 2018 · similarity 0.85)
- Learn to Pay Attention (ICLR 2018 · similarity 0.85)
- MaxViT: Multi-axis Vision Transformer (ECCV 2022 · similarity 0.84)
- Gated Convolutional Networks with Hybrid Connectivity for Image Classification (AAAI 2020 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
