AI Paper Atlas

Research map / Architectures and efficiency / Attention and transformers

Improving Vision Transformers by Revisiting High-Frequency Components

Jiawang Bai, Li Yuan, Shu-Tao Xia, Shuicheng Yan, Zhifeng Li, Wei Liu

ECCV 2022 · Architectures and efficiency › Attention and transformers · approach: Hybrid and interpretable ViT designs · 134 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
add HAT adversarial trainingViT-B · ImageNet-1K valtop-1 accuracy (%): 82.0 -> 83.2Improved
add HAT adversarial trainingViT-T · ImageNet-1K valtop-1 accuracy (%): 72.2 -> 73.3Improved
add HAT adversarial trainingSwin-B · ImageNet-1K valtop-1 accuracy (%): 83.5 -> 84.0Improved
add HAT adversarial trainingSwin-S · ImageNet-1K valtop-1 accuracy (%): 83.0 -> 83.3Improved
add HAT adversarial trainingVOLO-D5 (512 resolution) · ImageNet-1K valtop-1 accuracy (%): 87.1 -> 87.3Improved
add high-frequency adversarial perturbations (filter size 10) to training (ablation)ViT-B · ImageNet-1K valtop-1 accuracy (%): 82.0 -> 83.0Improved
add low-frequency adversarial perturbations to training (ablation)ViT-B · ImageNet-1K valtop-1 accuracy (%)No clear effect
combine HAT with knowledge distillationViT-B · ImageNet-1K valtop-1 accuracy (%): 83.2 -> 84.3Improved
train ViT-B with HAT instead of longer normal training at 2.3x cost (ablation)ViT-B · ImageNet-1K valtop-1 accuracy (%): 82.4 -> 83.2Improved
increase epsilon from 2/255 to 5/255 (ablation)HAT ViT-B · ImageNet-1K valtop-1 accuracy (%): 83.2 -> 83.0Hurt
add HAT with small training set (16K samples) (ablation)ViT-B · ImageNet subset (1/8 of training set)top-1 accuracy (%): +7.1Improved
add HAT to fine-tuning under SimMIM pre-trainingViT-B (SimMIM) · ImageNet-1K valtop-1 accuracy (%): 83.6 -> 83.9Improved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy