Research map / Vision-language and multimodal / Cross-modal learning
Tagging before Alignment: Integrating Multi-Modal Tags for Video-Text Retrieval
Yizhen Chen, Jie Wang, Lijian Lin, Zhongang Qi, Jin Ma, Ying Shan
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use multi-modal tags as alignment anchors (ablation) | TABLE · MSR-VTT, T2V | R@1: 45.2 -> 46.8 | Improved |
| Add object tags (ablation) | TABLE · MSR-VTT, T2V | R@1: 45.2 -> 45.6 | Improved |
| Add person tags to object tags (ablation) | TABLE · MSR-VTT, T2V | R@1: 45.6 -> 45.6 | No clear effect |
| Add motion tags to object and person tags (ablation) | TABLE · MSR-VTT, T2V | R@1: 45.6 -> 46.2 | Improved |
| Add scene tags to object, person, and motion tags (ablation) | TABLE · MSR-VTT, T2V | R@1: 46.2 -> 46.5 | Improved |
| Add audio tags to object, person, motion, and scene tags (ablation) | TABLE · MSR-VTT, T2V | R@1: 46.5 -> 46.8 | Improved |
| Add VTM auxiliary supervision (ablation) | TABLE · MSR-VTT, V2T | R@1: 45.9 -> 46.6 | Improved |
| Add MLM to VTM auxiliary supervision (ablation) | TABLE · MSR-VTT, T2V | R@1: 46.7 -> 47.1 | Improved |
| Add MLM to VTM auxiliary supervision (ablation) | TABLE · MSR-VTT, V2T | R@1: 46.6 -> 47.2 | Improved |
| Apply the CAMoE inference strategy (ablation) | TABLE · MSR-VTT, T2V | R@1: 47.1 -> 52.3 | Improved |
| Use TABLE instead of CAMoE | CLIP-based video-text retrieval · MSR-VTT, T2V | R@1: 47.1 vs 44.6 | Improved |
| Use TABLE instead of CAMoE | CLIP-based video-text retrieval · MSVD, T2V | R@1: 49.9 vs 46.9 | Improved |
Closest papers by meaning
- Towards Balanced Alignment: Modal-Enhanced Semantic Modeling for Video Moment Retrieval (AAAI 2024 · similarity 0.90)
- DiscoVLA: Discrepancy Reduction in Vision, Language, and Alignment for Parameter-Efficient Video-Text Retrieval (CVPR 2025 · similarity 0.89)
- Stitching Segments and Sentences towards Generalization in Video-Text Pre-training (AAAI 2024 · similarity 0.89)
- TOPA: Extending Large Language Models for Video Understanding via Text-Only Pre-Alignment (NeurIPS 2024 · similarity 0.88)
- VideoComp: Advancing Fine-Grained Compositional and Temporal Alignment in Video-Text Models (CVPR 2025 · similarity 0.88)
- VidLA: Video-Language Alignment at Scale (CVPR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
