Research map / Video / Video-language understanding
TS2-Net: Token Shift and Selection Transformer for Text-Video Retrieval
Yuqi Liu, Pengfei Xiong, Luhui Xu, Shengming Cao, Qin Jin
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add token shift in layers 11-12 (25%) to baseline (ablation) | CLIP-based baseline · MSR-VTT 1k-A (text-to-video) | R@1: 45.4 -> 46.2; rsum 401.5 -> 406.2 | Improved |
| apply token shift to all layers 1-12 instead of only 11-12 (ablation) | Token Shift · MSR-VTT 1k-A (text-to-video) | R@1: 46.2 -> 42.8 | Hurt |
| apply token shift to layers 9-12 instead of 11-12 (ablation) | Token Shift · MSR-VTT 1k-A (text-to-video) | R@1: 46.2 -> 45.2 | Hurt |
| shift 12.5% or 50% of tokens instead of 25% (ablation) | Token Shift · MSR-VTT 1k-A (text-to-video) | R@1: 46.0 and 46.1 vs 46.2 | No clear effect |
| replace channel shift with token shift (ablation) | Shift variants · MSR-VTT 1k-A (text-to-video) | R@1: 45.6 -> 46.2 | Improved |
| replace [CLS] channel shift with token shift (ablation) | Shift variants · MSR-VTT 1k-A (text-to-video) | R@1: 45.8 -> 46.2 | Improved |
| add learned token selection (top-4) after token shift (ablation) | TS2-Net · MSR-VTT 1k-A (text-to-video) | R@1: 46.2 -> 47.0; rsum 406.2 -> 408.4 | Improved |
| use all 50 tokens instead of top-K selection (ablation) | Token Selection · MSR-VTT 1k-A (text-to-video) | R@1: 47.0 -> 45.8 | Hurt |
| replace learned selection with random selection of 4 tokens (ablation) | Token Selection · MSR-VTT 1k-A (text-to-video) | R@1: 47.0 -> 46.4 | Hurt |
| increase selected tokens from 4 to 8 (ablation) | Token Selection · MSR-VTT 1k-A (text-to-video) | R@1: 47.0 -> 46.4 | Hurt |
| apply inverted softmax at inference | TS2-Net · MSR-VTT 1k-A (text-to-video) | R@1: 47.0 -> 51.1 | Improved |
| TS2-Net vs CLIP4Clip on hard subset of small objects and subtle motion | TS2-Net · MSR-VTT 1k-A hard subset (103 queries) | R@1: 39.8 -> 79.6 | Improved |
Later papers that compared with this method
10 later accepted papers report TS2-Net as a baseline in their results tables; 10 of them report a better number on most of the results they share with it.
- On Learning Multi-Modal Forgery Representation for Diffusion Generated Video Detection (NeurIPS 2024 · better on 1 of 1 shared result)
- Hierarchical Banzhaf Interaction for General Video-Language Representation Learning (TPAMI 2024 · better on 1 of 1 shared result)
- Progressive Spatio-Temporal Prototype Matching for Text-Video Retrieval (ICCV 2023 · better on 7 of 7 shared results)
- Unified Coarse-to-Fine Alignment for Video-Text Retrieval (ICCV 2023 · better on 2 of 3 shared results)
- UATVR: Uncertainty-Adaptive Text-Video Retrieval (ICCV 2023 · better on 3 of 3 shared results)
- PIDRo: Parallel Isomeric Attention with Dynamic Routing for Text-Video Retrieval (ICCV 2023 · better on 3 of 3 shared results)
- Video-Text As Game Players: Hierarchical Banzhaf Interaction for Cross-Modal Representation Learning (CVPR 2023 · better on 2 of 2 shared results)
- Cap4Video: What Can Auxiliary Captions Do for Text-Video Retrieval? (CVPR 2023 · better on 1 of 1 shared result)
2 more not listed.
Matched by method name in results tables read by AI. Authors' own tables tend to favour the newer method, and a comparison can be missed when a method is renamed.
Closest papers by meaning
- Efficient Video Transformers with Spatial-Temporal Token Selection (ECCV 2022 · similarity 0.88)
- TempMe: Video Temporal Token Merging for Efficient Text-Video Retrieval (ICLR 2025 · similarity 0.87)
- Video-ColBERT: Contextualized Late Interaction for Text-to-Video Retrieval (CVPR 2025 · similarity 0.86)
- Towards Efficient and Effective Text-to-Video Retrieval with Coarse-to-Fine Visual Representation Learning (AAAI 2024 · similarity 0.86)
- FlexSelect: Flexible Token Selection for Efficient Long Video Understanding (NeurIPS 2025 · similarity 0.85)
- Mitigating Semantic Collapse in Partially Relevant Video Retrieval (NeurIPS 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
