Research map / Detection and segmentation / CNN features
Swin Transformer V2: Scaling Up Capacity and Resolution
Check your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace pre-norm with res-post-norm (ablation) | Swin Transformer · ImageNet-1K | top-1 accuracy: 81.5 -> 81.6; 83.2 -> 83.3; 83.6 -> 83.8 | Helped |
| add scaled cosine attention to res-post-norm (ablation) | Swin Transformer · ImageNet-1K | top-1 accuracy: 81.6 -> 81.7; 83.3 -> 83.6; 83.8 -> 84.1 | Helped |
| replace pre-norm with sandwich normalization (ablation) | Swin Transformer · ImageNet-1K | top-1 accuracy: 83.2 -> 82.6 | Hurt |
| replace pre-norm with post-norm (ablation) | Swin Transformer · ImageNet-1K | top-1 accuracy: 83.2 -> 83.3; 83.6 -> 83.6 | No clear effect |
| replace parameterized position bias with continuous position bias (ablation) | Swin-T · ImageNet-1K at larger image and window sizes | top-1 accuracy: 79.4/82.7 -> 82.0/82.9 | Helped |
| use log-spaced instead of linear-spaced coordinates for CPB (ablation) | Swin-T · ImageNet-1K at larger image and window sizes | top-1 accuracy: 82.0/82.9 -> 82.4/83.2 | Helped |
| replace parameterized position bias with Log-CPB (ablation) | SwinV2-S · ImageNet-1K W16, I512 | top-1 accuracy: 79.4/84.9 -> 82.9/85.4 | Helped |
| replace parameterized position bias with Log-CPB (ablation) | SwinV2-B · ImageNet-1K W16, I512 | top-1 accuracy: 81.0/85.3 -> 83.8/85.6 | Helped |
| increase model capacity to SwinV2-G | SwinV2 · COCO test-dev | box AP: +1.8 | Helped |
| increase model capacity to SwinV2-G | SwinV2 · ADE20K val | mIoU: +1.5 | Helped |
| use a larger window at test time | SwinV2-G · ADE20K val | mIoU: +0.2 | Helped |
| use a larger window at test time | Video-SwinV2-G · Kinetics-400 | top-1 accuracy: +0.2% | Helped |
Closest papers by meaning
- Video Swin Transformer (CVPR 2022 · similarity 0.89)
- Win-Win: Training High-Resolution Vision Transformers from Two Windows (ICLR 2024 · similarity 0.86)
- SLAB: Efficient Transformers with Simplified Linear Attention and Progressive Re-parameterized Batch Normalization (ICML 2024 · similarity 0.86)
- TGBFormer: Transformer-GraphFormer Blender Network for Video Object Detection (AAAI 2025 · similarity 0.85)
- ScaleKD: Strong Vision Transformers Could Be Excellent Teachers (NeurIPS 2024 · similarity 0.85)
- ParameterNet: Parameters Are All You Need for Large-scale Visual Pretraining of Mobile Networks (CVPR 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.