AI Paper Atlas

Research map / Architectures and efficiency / Neural architecture search

GLiT: Neural Architecture Search for Global and Local Image Transformer

Boyu Chen, Peixia Li, Chuming Li, Baopu Li, Lei Bai, Chen Lin, Ming Sun, Junjie Yan, Wanli Ouyang

ICCV 2021 · Architectures and efficiency › Neural architecture search · approach: Bi-level differentiable NAS · 105 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
replace 1 of 3 self-attention heads with a Conv1d head (ablation)DeiT-Tiny · ImageNettop-1 accuracy (%): 72.20 -> 72.89Improved
replace 2 of 3 self-attention heads with Conv1d heads (ablation)DeiT-Tiny · ImageNettop-1 accuracy (%): 72.20 -> 73.98Improved
replace all self-attention heads with Conv1d heads (ablation)DeiT-Tiny · ImageNettop-1 accuracy (%): 72.20 -> 71.02Hurt
search global-local distribution per block (ablation)DeiT-Tiny · ImageNettop-1 accuracy (%): 72.2 -> 75.4Improved
search the detailed sub-module architecture on top of distribution (ablation)GLiT-Tiny · ImageNettop-1 accuracy (%): 75.4 -> 76.3Improved
searched distribution vs best uniform human-designed global-local mix (ablation)GLiT-Tiny · ImageNettop-1 accuracy (%): 73.98 -> 75.4Improved
use GLiT search space instead of NLP-NAS search space (ablation)GLiT-Tiny · ImageNettop-1 accuracy (%): 73.4 -> 76.3Improved
use hierarchical search instead of SPOS (ablation)GLiT-Tiny · ImageNettop-1 accuracy (%): 72.7 -> 76.3Improved
use hierarchical search instead of random search (ablation)GLiT-Tiny · ImageNettop-1 accuracy (%): 73.3 -> 76.3Improved
use Conv1D rather than local-area self-attention for locality (ablation)GLiT-Tiny · ImageNettop-1 accuracy (%): 72.4 -> 76.3Improved
use Conv2D instead of Conv1D (ablation)GLiT-Tiny · ImageNettop-1 accuracy (%): 76.3 -> 76.4No clear effect
GLiT-Tiny vs DeiT-Tiny under similar FLOPsDeiT-Tiny · ImageNettop-1 accuracy (%): 72.2 -> 76.3Improved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy