Research map / Architectures and efficiency / Attention and transformers
SpatialFormer: Towards Generalizable Vision Transformers with Explicit Spatial Understanding
Han Xiao, Wenzhao Zheng, Sicheng Zuo, Peng Gao, Jie Zhou, Jiwen Lu
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add adaptive spatial tokens initialized with positional encoding (ablation) | SpatialFormer-S · ImageNet-1k val 224 | top-1 accuracy: 81.3 -> 83.8 | Improved |
| increase spatial tokens from 4x4 to 8x8 (ablation) | SpatialFormer-S · ImageNet-1k val 224 | top-1 accuracy: 83.0 -> 83.8 | Improved |
| increase spatial tokens from 8x8 to 16x16 (ablation) | SpatialFormer-S · ImageNet-1k val 224 | top-1 accuracy: 83.8 -> 84.1 (FLOPs 4.8G -> 7.0G) | Improved |
| use BCA blocks in more stages instead of self-attention in last two stages (ablation) | SpatialFormer-S · ImageNet-1k val 224 | top-1 accuracy: 83.8 -> 84.1 (params 25M -> 37M) | Improved |
| replace conventional positional embeddings with spatial tokens (ablation) | SpatialFormer-S · ImageNet-1k val 224 | top-1 accuracy: 82.1 -> 83.8 (best conventional PE vs spatial tokens) | Improved |
| use 3D positional encodings to initialize spatial tokens instead of 2D (ablation) | SpatialFormer with BEVFormer · nuScenes val | NDS: 0.389 -> 0.392 | Improved |
| replace Swin-T backbone with SpatialFormer in BEVFormer | BEVFormer · nuScenes val | NDS: 0.369 -> 0.389 | Improved |
| transfer 224-trained model to other resolutions without fine-tuning | SpatialFormer vs DeiT · ImageNet resolution generalization | top-1 accuracy | Improved |
| use spatial tokens as initial queries in Deformable DETR (two-stage) | SpatialFormer-S · COCO val | box AP: 43.5 -> 46.5 | Improved |
Closest papers by meaning
- SpatioLM: Towards General Physical Spatial Intelligence in Vision-Language Models (ICML 2026 · similarity 0.87)
- SpatialRGPT: Grounded Spatial Reasoning in Vision-Language Models (NeurIPS 2024 · similarity 0.87)
- SpatialLadder: Progressive Training for Spatial Reasoning in Vision-Language Models (ICLR 2026 · similarity 0.87)
- Spatial-MLLM: Boosting MLLM Capabilities in Visual-based Spatial Intelligence (NeurIPS 2025 · similarity 0.86)
- Spatial-SSRL: Enhancing Spatial Understanding via Self-Supervised Reinforcement Learning (CVPR 2026 · similarity 0.86)
- Win-Win: Training High-Resolution Vision Transformers from Two Windows (ICLR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
