Research map / Architectures and efficiency / Attention and transformers
RegionViT: Regional-to-Local Attention for Vision Transformers
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add regional tokens (regional-to-local attention branch) (ablation) | RegionViT-S · ImageNet1K | Top-1 accuracy: +0.4 (82.2 to 82.6) | Helped |
| add regional tokens (ablation) | RegionViT-S · MS COCO RetinaNet 1x / MaskRCNN 1x / ADE20K SemanticFPN | AP / mIoU: RetinaNet AP +1.5 (40.7 to 42.2); MaskRCNN APb/APm +2.0/+1.7; mIoU +1.4 (42.3 to 43.7) | Helped |
| use 3x3 depthwise conv for downsampling instead of 2x2 conv (ablation) | RegionViT-S · ImageNet1K | Top-1 accuracy: +0.3 vs 2x2 conv (82.3 to 82.6) | Helped |
| use regular 3x3 convolution for downsampling instead of depthwise (ablation) | RegionViT-S · ImageNet1K | Top-1 accuracy: 82.5 vs 82.6 with more params and FLOPs | No clear effect |
| use separate weights for RSA and LSA instead of sharing (ablation) | RegionViT-S · ImageNet1K | Top-1 accuracy: +0.1 (82.6 to 82.7) with params 30.6M to 40.4M | No clear effect |
| add absolute position embedding on top of relative position bias (ablation) | RegionViT-S · ImageNet1K | Top-1 accuracy: +0.1 (82.6 to 82.7), but fixes input resolution | No clear effect |
| use relative position bias instead of absolute position embedding (ablation) | RegionViT-S · ImageNet1K | Top-1 accuracy: +0.2 (82.4 to 82.6) | Helped |
| use overlapped windows (50% overlap) instead of non-overlapped (ablation) | RegionViT-S · ImageNet1K | Top-1 accuracy: +0.2 (82.6 to 82.8) at 3.5x FLOPs (5.3G to 18.8G) | No clear effect |
| replace R2L attention with shifted-window attention (ablation) | Swin-style pyramid model · ImageNet1K | Top-1 accuracy: -0.7 (82.0 R2L to 81.3 shifted-window) | Hurt |
| use all regional tokens in LSA instead of only the corresponding one (ablation) | RegionViT-S · ImageNet1K | Top-1 accuracy: +0.1 (82.6 to 82.7) with 13% more FLOPs | No clear effect |
| increase window size from 7 to 14 (RegionViT-S+ and larger variants) | RegionViT-S · ImageNet1K | Top-1 accuracy: +0.7 (82.6 to 83.3) | Helped |
| add PEG (position encoding generator) to RegionViT-S+ | RegionViT-S+ · COCO val2017 1x MaskRCNN | APb: +0.7 (43.5 to 44.2) | Helped |
Closest papers by meaning
- On the Connection between Local Attention and Dynamic Depth-wise Convolution (ICLR 2022 · similarity 0.91)
- You Only Need Less Attention at Each Stage in Vision Transformers (CVPR 2024 · similarity 0.91)
- Attention Guided CAM: Visual Explanations of Vision Transformer Guided by Self-Attention (AAAI 2024 · similarity 0.89)
- FasterViT: Fast Vision Transformers with Hierarchical Attention (ICLR 2024 · similarity 0.89)
- LF-ViT: Reducing Spatial Redundancy in Vision Transformer for Efficient Image Recognition (AAAI 2024 · similarity 0.89)
- Recognize Any Regions (NeurIPS 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.