Research map / Video / Video segmentation and tracking
TMT-VIS: Taxonomy-aware Multi-dataset Joint Training for Video Instance Segmentation
rongkun Zheng, Lu Qi, Xi Chen, Yi Wang, Kun Wang, Yu Qiao, Hengshuang Zhao
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add taxonomy compilation and injection to Mask2Former-VIS | Mask2Former-VIS · YouTube-VIS 2019 val | AP: 46.4 -> 49.7 | Improved |
| Add taxonomy compilation and injection to Mask2Former-VIS | Mask2Former-VIS · YouTube-VIS 2021 val | AP: 40.6 -> 44.9 | Improved |
| Add taxonomy compilation and injection to Mask2Former-VIS | Mask2Former-VIS · OVIS val | AP: 16.5 -> 22.8 | Improved |
| Add taxonomy compilation and injection to Mask2Former-VIS | Mask2Former-VIS · UVO val | AP: 18.2 -> 21.2 | Improved |
| Add taxonomy-aware matching loss (ablation) | TMT-VIS · YouTube-VIS 2019 val | AP: 47.0 -> 47.3 | Improved |
| Add taxonomy-aware matching loss (ablation) | TMT-VIS · YouTube-VIS 2019 val, multiple datasets | AP: 49.2 -> 49.7 | Improved |
| Use taxonomy embedding set size 10 rather than 1 (ablation) | TMT-VIS · YouTube-VIS 2019 val, multiple datasets | AP: 47.1 -> 49.7 | Improved |
| Increase taxonomy embedding set size from 10 to 100 (ablation) | TMT-VIS · YouTube-VIS 2019 val, multiple datasets | AP: 49.7 -> 47.2 | Hurt |
| Replace cross-attention with addition for taxonomy injection (ablation) | TMT-VIS · YouTube-VIS 2019 val, multiple datasets | AP: 49.7 -> 47.6 | Hurt |
| Replace cross-attention with concatenation for taxonomy injection (ablation) | TMT-VIS · YouTube-VIS 2019 val, multiple datasets | AP: 49.7 -> 47.4 | Hurt |
| Use sampling ratio 1:1:0.75 instead of 2:1:0.75 (ablation) | TMT-VIS · YouTube-VIS 2019 val, multiple datasets | AP: 48.6 -> 49.7 | Improved |
| Use sampling ratio 1:1:1 instead of 1:1:0.75 (ablation) | TMT-VIS · YouTube-VIS 2019 val, multiple datasets | AP: 49.7 -> 49.5 | Hurt |
Closest papers by meaning
- Towards Open-Vocabulary Video Instance Segmentation (ICCV 2023 · similarity 0.90)
- Video Mask Transfiner for High-Quality Video Instance Segmentation (ECCV 2022 · similarity 0.89)
- End-to-End Video Instance Segmentation With Transformers (CVPR 2021 · similarity 0.88)
- Unified Embedding Alignment for Open-Vocabulary Video Instance Segmentation (ECCV 2024 · similarity 0.88)
- Minimizing Labeled, Maximizing Unlabeled: An Image-Driven Approach for Video Instance Segmentation (CVPR 2025 · similarity 0.88)
- A Generalized Framework for Video Instance Segmentation (CVPR 2023 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
