Research map / Detection and segmentation / CNN features
XCiT: Cross-Covariance Image Transformers
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use XCiT rather than a ViT-B/8 backbone | DINO self-supervised learning · ImageNet-1k, 300 epochs | linear accuracy (%): 80.1 -> 80.3 | Helped |
| Use 8×8 rather than 16×16 patches | XCiT backbones · COCO mini-val, Mask R-CNN | box AP: 42.7 -> 44.5 | Helped |
| Use 8×8 rather than 16×16 patches | XCiT backbones · ADE20k val, Semantic FPN | mIoU: 43.9 -> 44.2 | Helped |
| Replace convolutional patch projection with linear patch projection (ablation) | XCiT-S12 · ImageNet-1k | top-1 accuracy (%): 82.0 -> 81.1 | Hurt |
| Add the LPI layer (ablation) | XCiT-S12/16 · ImageNet-1k | top-1 accuracy (%): 80.8 -> 82.0 | Helped |
| Add the XCA layer (ablation) | XCiT-S12/16 · ImageNet-1k | top-1 accuracy (%): 75.9 -> 82.0 | Helped |
| Remove L2 normalization of queries and keys (ablation) | XCiT-S12/16 · ImageNet-1k | training convergence | Hurt |
| Remove the learned temperature parameter (ablation) | XCiT-S12/16 · ImageNet-1k | top-1 accuracy (%): 82.0 -> 81.8 | Hurt |
| Replace full attention with efficient attention variants | DeiT-S · ImageNet-1k | top-1 accuracy: 79.9 -> 73.5 (SRA average pool); 79.9 -> 74.0 (SRA convolutional); 79.9 -> 75.7 (Linformer); 79.9 -> 76.3 (Efficient Tra | Hurt |
| Use self-supervised DINO pretraining rather than supervised pretraining | XCiT-S12/16 and XCiT-S12/8 retrieval descriptors · Image retrieval, Holidays and ROxford5k | mAP: 86.0 -> 87.3 (Holidays, XCiT-S12/16); 30.1 -> 35.1 (ROxford5k medium, XCiT-S12/16); 86.4 -> 88.3 (Holidays, XCiT-S12/8); | Mixed |
Closest papers by meaning
- CrossFormer: A Versatile Vision Transformer Hinging on Cross-scale Attention (ICLR 2022 · similarity 0.89)
- XAttention: Block Sparse Attention with Antidiagonal Scoring (ICML 2025 · similarity 0.86)
- CAT: Circular-Convolutional Attention for Sub-Quadratic Transformers (NeurIPS 2025 · similarity 0.86)
- Linear Differential Vision Transformer: Learning Visual Contrasts via Pairwise Differentials (NeurIPS 2025 · similarity 0.85)
- xT: Nested Tokenization for Larger Context in Large Images (ICML 2024 · similarity 0.85)
- Sample- and Parameter-Efficient Auto-Regressive Image Models (CVPR 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.