Research map / Architectures and efficiency / Attention and transformers
CSWin Transformer: A General Vision Transformer Backbone With Cross-Shaped Windows
Check your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace parallel multi-head grouping with sequential attention (ablation) | CSWin-T · ImageNet-1K | Top-1 accuracy: 82.7 -> 82.4 | Hurt |
| Replace parallel multi-head grouping with sequential attention (ablation) | CSWin-T · COCO | box AP: 46.7 -> 45.1 | Hurt |
| Replace parallel multi-head grouping with sequential attention (ablation) | CSWin-T · ADE20K | mIoU: 48.2 -> 46.2 | Hurt |
| Increase stripe width, using stage-varying widths rather than fixed width 1 (ablation) | CSWin-T · ADE20K | mIoU: 47.5 -> 48.2 | Helped |
| Use cross-shaped window attention rather than sequential axial attention (ablation) | CSWin-T backbone · ImageNet-1K | Top-1 accuracy: 81.5 -> 82.2 | Helped |
| Use cross-shaped window attention rather than sequential axial attention (ablation) | CSWin-T backbone · COCO | box AP: 40.4 -> 43.4 | Helped |
| Use cross-shaped window attention rather than sequential axial attention (ablation) | CSWin-T backbone · ADE20K | mIoU: 39.8 -> 43.4 | Helped |
| Add LePE instead of using no positional encoding (ablation) | CSWin-T · ADE20K | mIoU: 47.0 -> 48.2 | Helped |
| Use LePE rather than RPE (ablation) | CSWin-T · COCO | box AP: 45.5 -> 46.7 | Helped |
| Use LePE rather than RPE (ablation) | CSWin-T · ADE20K | mIoU: 46.6 -> 48.2 | Helped |
| Pretrain on ImageNet-21K rather than ImageNet-1K | CSWin-B · ImageNet-1K | Top-1 accuracy: 1.6%~1.7% gain | Helped |
| Increase stripe width (ablation) | CSWin Transformer · ImageNet-1K | Top-1 accuracy | Helped |
Closest papers by meaning
- Pale Transformer: A General Vision Transformer Backbone with Pale-Shaped Attention (AAAI 2022 · similarity 0.89)
- CrossFormer: A Versatile Vision Transformer Hinging on Cross-scale Attention (ICLR 2022 · similarity 0.88)
- An Image is Worth More Than 16x16 Patches: Exploring Transformers on Individual Pixels (ICLR 2025 · similarity 0.86)
- You Only Need Less Attention at Each Stage in Vision Transformers (CVPR 2024 · similarity 0.86)
- Win-Win: Training High-Resolution Vision Transformers from Two Windows (ICLR 2024 · similarity 0.86)
- Rethinking Decoders for Transformer-based Semantic Segmentation: A Compression Perspective (NeurIPS 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.