AI Paper Atlas

Research map / Architectures and efficiency / Attention and transformers

Hard-Attention for Scalable Image Classification

Athanasios Papadopoulos, Pawel Korus, Nasir Memon

NeurIPS 2021 · Architectures and efficiency › Attention and transformers · approach: Sparse and structured attention · 10 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
Use TNet hard attention instead of the fully convolutional BagNet-77 baselineImageNet classification · ImageNet ILSVRC 2012 validationTop-1 accuracy: 73.42 % -> 74.12 %Improved
Use TNet with 1 attended location instead of Saccader with 30 locationsImageNet classification · ImageNet ILSVRC 2012 validationTop-1 accuracy: 72.27 % -> 73.12 %Improved
Use TNet with 1 attended location instead of BagNet-77ImageNet classification · ImageNet ILSVRC 2012 validationFLOPs: 18.42 -> 3.63Improved
Use TNet hard attention at 448² rather than resizing the whole image for EfficientNet-B0fMoW classification · fMoWTop-1 accuracy: 69.83 % -> 71.46 %Improved
Use TNet hard attention at 448² rather than resizing the whole image for EfficientNet-B0fMoW classification · fMoWFLOPs: 1.54 -> 1.16Improved
Use TNet with 4 locations at 896² rather than resizing the whole image for EfficientNet-B0fMoW classification · fMoWTop-1 accuracy: 70.6 % -> 72.16 %Improved
Use TNet with 4 locations at 896² rather than resizing the whole image for EfficientNet-B0fMoW classification · fMoWFLOPs: 6.18 -> 1.94Improved
Add per-feature regularization (ablation)TNet · ImageNet ILSVRC 2012 validation, 1 locationTop-1 accuracy: 65.21 % -> 73.12 %Improved
Add per-feature regularization (ablation)TNet · ImageNet ILSVRC 2012 validation, 3 locationsTop-1 accuracy: 67.33 % -> 74.41 %Improved
Add per-feature regularization (ablation)TNet · ImageNet ILSVRC 2012 validation, 5 locationsTop-1 accuracy: 68.55 % -> 74.62 %Improved
Include the global contextual feature alongside attended-location features (ablation)TNet · ImageNet ILSVRC 2012 validation, 2 locationsTop-1 accuracy: 67.95 % -> 74.12 %Improved
Include the global contextual feature alongside attended-location features (ablation)TNet · ImageNet ILSVRC 2012 validation, 3 locationsTop-1 accuracy: 69.66 % -> 74.41 %Improved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy