AI Paper Atlas

Research map / Architectures and efficiency / Attention and transformers

Saccader: Improving Accuracy of Hard Attention Models for Vision

Gamaleldin Elsayed, Simon Kornblith, Quoc V Le

NeurIPS 2019 · Architectures and efficiency › Attention and transformers · approach: Sparse and structured attention · 37 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
Pretrain the location network using class-label-derived sorted-logit locations before policy-gradient training (ablation)Saccader · ImageNetclassification accuracyImproved
Add the attention network to the location-selection system (ablation)Saccader · ImageNetclassification accuracyImproved
Add the Saccader cell rather than selecting locations with the ordered-logits policy (ablation)BagNet-77-lowD · ImageNetclassification accuracyImproved
Use the learned Saccader policy instead of the ordered-logits policyBagNet-77-lowD · ImageNetclassification accuracyImproved
Use the learned Saccader policy instead of DRAMImageNet hard-attention classification · ImageNetclassification accuracyImproved
Allow DRAM to use more glimpses than it was trained onDRAM · ImageNetclassification accuracyHurt
Add a separate NASNet classification network for extracted patchesSaccader · ImageNet 224×224classification accuracyImproved
Use 331×331 images for patch classification instead of 224×224 imagesSaccader-NASNet · ImageNetclassification accuracyImproved
Occlude the patches selected by SaccaderResNet-v2-50 classifier · ImageNet validationclassification accuracyHurt
Use hard attention without adversarial trainingSaccader and other tested attention models · ImageNet validation subset, ε = 2/255adversarial accuracyNo clear effect

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy