Research map / Detection and segmentation / CNN features
CrossFormer: A Versatile Vision Transformer Hinging on Cross-scale Attention
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace single-scale CEL with cross-scale embeddings (ablation) | CrossFormer-S · ImageNet val | accuracy: 81.5% -> 82.5% | Helped |
| Use overlapping Stage-1 patches with an 8×8 rather than 4×4 single-scale kernel (ablation) | CrossFormer-S · ImageNet val | accuracy: 81.5% -> 81.9% | Helped |
| Replace LSDA with PVT-like attention (ablation) | CrossFormer-S with CEL · ImageNet val | accuracy: 82.5% -> 81.3% | Hurt |
| Replace LSDA with Swin-like local attention (ablation) | CrossFormer-S with CEL · ImageNet val | accuracy: 82.5% -> 81.9% | Hurt |
| Remove CEL (ablation) | CrossFormer-S with LSDA · ImageNet val | accuracy: 82.5% -> 81.5% | Hurt |
| Replace DPB with absolute position embeddings (ablation) | CrossFormer-S · ImageNet val | accuracy: 82.5% -> 82.1% | Hurt |
| Add residual connections to DPB (ablation) | CrossFormer-S · ImageNet val | accuracy: 82.5% -> 82.4% | Hurt |
| Increase group sizes and intervals in the first two stages (ablation) | CrossFormer-S · COCO 2017 val, RetinaNet 1× | box AP: 44.4 -> 44.2 | Hurt |
| Increase group sizes and intervals in the first two stages (ablation) | CrossFormer-S · ADE20K val, Semantic FPN 80K | IOU: 46.0 -> 46.4 | Helped |
| Use DPB rather than absolute position embeddings (ablation) | CrossFormer-S · ImageNet val | accuracy: 82.1% -> 82.5% | Helped |
| Use cross-scale attention architecture | CrossFormer · COCO 2017 val, RetinaNet 1× | box AP: +1.0 | Helped |
| Use cross-scale attention architecture | CrossFormer · ADE20K val, Semantic FPN 80K | IOU: +4.4 | Helped |
Closest papers by meaning
- Linear Differential Vision Transformer: Learning Visual Contrasts via Pairwise Differentials (NeurIPS 2025 · similarity 0.89)
- Data-Adaptive Mahalanobis Metric Learning for Cross-Head Attention in Transformers (NeurIPS 2026 · similarity 0.87)
- Skip-Attention: Improving Vision Transformers by Paying Less Attention (ICLR 2024 · similarity 0.87)
- Breaking the Low-Rank Dilemma of Linear Attention (CVPR 2025 · similarity 0.87)
- CARE Transformer: Mobile-Friendly Linear Visual Transformer via Decoupled Dual Interaction (CVPR 2025 · similarity 0.87)
- You Only Need Less Attention at Each Stage in Vision Transformers (CVPR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.