Research map / Video / Video segmentation and tracking
Temporally Efficient Vision Transformer for Video Instance Segmentation
Check your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add multi-scale training | TeViT · YouTube-VIS-2019 | AP: 45.9 -> 46.6 | Helped |
| add messenger shift mechanism (ablation) | TeViT baseline without temporal components · YouTube-VIS-2019 | AP: 42.5 -> 43.1; +0.6 | Helped |
| add spatiotemporal query interaction (ablation) | TeViT baseline without temporal components · YouTube-VIS-2019 | AP: 42.5 -> 45.2; +2.7 | Helped |
| add messenger shift and STQI together (ablation) | TeViT baseline without temporal components · YouTube-VIS-2019 | AP: 42.5 -> 45.9; +3.4 | Helped |
| replace spatial-only query interaction with fused space-time MHSA (ablation) | QueryInst-based video instance head · YouTube-VIS-2019 | AP: 43.1 -> 43.9; +0.8 | Helped |
| use parameter-shared STQI instead of spatial-only interaction (ablation) | QueryInst-based video instance head · YouTube-VIS-2019 | AP: 43.1 -> 45.9; +2.7 | Helped |
| replace messenger shift with extra MHSA and FFN on messenger tokens (ablation) | TeViT · YouTube-VIS-2019 | AP: 45.9 -> 44.5; 1.4 AP | Hurt |
| replace messenger shift with convolution for frame-level aggregation (ablation) | TeViT · YouTube-VIS-2019 | AP: 45.9 -> 41.8 | Hurt |
| increase messenger tokens from 8 to 32 (ablation) | TeViT · YouTube-VIS-2019 | AP: 45.3 -> 45.9 | Helped |
| increase training clip length from 2 to 7 frames (ablation) | TeViT · YouTube-VIS-2019 | AP: 41.1 -> 46.3 | Helped |
| remove spatial MHSA from the STQI head (ablation) | TeViT with ResNet-50 backbone · YouTube-VIS-2019 | AP: 41.7 -> 36.8 | Hurt |
| replace learned messenger tokens with zeros at inference (ablation) | TeViT · YouTube-VIS-2019 | AP: 45.9 -> 45.5; -0.4 | No clear effect |
Closest papers by meaning
- InstanceFormer: An Online Video Instance Segmentation Framework (AAAI 2023 · similarity 0.90)
- Referred by Multi-Modality: A Unified Temporal Transformer for Video Object Segmentation (AAAI 2024 · similarity 0.86)
- SyncVIS: Synchronized Video Instance Segmentation (NeurIPS 2024 · similarity 0.86)
- Adapting Short-Term Transformers for Action Detection in Untrimmed Videos (CVPR 2024 · similarity 0.85)
- MobileInst: Video Instance Segmentation on the Mobile (AAAI 2024 · similarity 0.85)
- SHViT: Single-Head Vision Transformer with Memory Efficient Macro Design (CVPR 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.