Research map / Architectures and efficiency / Attention and transformers
Multiscale Vision Transformers
Haoqi Fan, Bo Xiong, Karttikeya Mangalam, Yanghao Li, Zhicheng Yan, Jitendra Malik, Christoph Feichtenhofer
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add a single Q-pooling scale stage (resolution reduction) after 6 blocks (ablation) | ViT-B · Kinetics-400 val | top-1 accuracy: 68.5 -> 71.0 | Improved |
| add K,V pooling (ablation) | ViT-B · Kinetics-400 val | top-1 accuracy: 68.5 -> 69.1 | Improved |
| use learnable conv pooling instead of max pooling (ablation) | MViT-B 16x4 · Kinetics-400 val | top-1 accuracy: 77.2 -> 78.3 | Improved |
| use average pooling instead of max pooling (ablation) | MViT-B 16x4 · Kinetics-400 val | top-1 accuracy: 77.2 -> 75.4 | Hurt |
| use overlapping pooling kernel s+1 instead of kernel equal to stride (ablation) | MViT-B 16x4 · Kinetics-400 val | top-1 accuracy: 76.1 -> 77.2 | Improved |
| add layer-normalized skip connection at stage transitions instead of unnormalized (ablation) | MViT-B 16x4 · Kinetics-400 val | top-1 accuracy: 74.6 -> 77.2 | Improved |
| add skip connection at stage transitions (vs none) (ablation) | MViT-B 16x4 · Kinetics-400 val | top-1 accuracy: 74.7 -> 77.2 | Improved |
| add separate space and time positional embedding instead of none (ablation) | MViT-B 16x4 · Kinetics-400 val | top-1 accuracy: 75.8 -> 77.2 | Improved |
| use adaptive K,V pooling stride instead of fixed 2x4x4 (ablation) | MViT-B 16x4 · Kinetics-400 val | top-1 accuracy: 74.8 -> 77.1 | Improved |
| add K,V pooling (adaptive 1x8x8) instead of none (ablation) | MViT-B 16x4 · Kinetics-400 val | top-1 accuracy: 77.6 -> 77.2 | No clear effect |
| use temporal cubes (c_T=3) instead of 1x7x7 patches in cube1 (ablation) | MViT-B · Kinetics-400 val | top-1 accuracy: 75.6 -> 75.9 | Improved |
| sample 16 frames with temporal stride 2 instead of 8 frames (ablation) | MViT-B · Kinetics-400 val | top-1 accuracy: 75.9 -> 77.2 | Improved |
Closest papers by meaning
- MViTv2: Improved Multiscale Vision Transformers for Classification and Detection (CVPR 2022 · similarity 0.93)
- MaxViT: Multi-axis Vision Transformer (ECCV 2022 · similarity 0.89)
- Multiview Transformers for Video Recognition (CVPR 2022 · similarity 0.89)
- A Simple Single-Scale Vision Transformer for Object Detection and Instance Segmentation (ECCV 2022 · similarity 0.89)
- Scaling Vision Transformers (CVPR 2022 · similarity 0.89)
- An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale (ICLR 2021 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
