Research map / Video / Video-language understanding
Mitigating Modality and Language-Style Gaps for Zero-Shot Video Moment Retrieval
Jihyun Lee, Cheol-Ho Cho, Woojin Jun, Woojin Jeong, Jae-Pil Heo
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace mean span scoring with top-k query-matching score (QMS) (ablation) | Self-SiMS · QVHighlights val | R1@0.5: 55.7 -> 59.5 | Improved |
| add self-matching span score (SMS) on QMS (ablation) | Self-SiMS · QVHighlights val | R1@0.5: 59.5 -> 60.3 | Improved |
| add MLLM yes/no re-ranking (ablation) | Self-SiMS · QVHighlights val | mAP@avg: 38.7 -> 39.3 | Improved |
| generate spans from self-similarity instead of sliding window (ablation) | Self-SiMS · QVHighlights val | mAP@avg: 12.5 -> 39.3 | Improved |
| generate spans from self-similarity instead of k-means clustering (ablation) | Self-SiMS · QVHighlights val | mAP@avg: 31.3 -> 39.3 | Improved |
| use DINOv2 instead of CLIP-B/32 frame features for self-similarity (ablation) | Self-SiMS · QVHighlights val | mAP@avg: 35.3 -> 38.7 | Improved |
| add SMS across CLIP-B/32 encoder (ablation) | Self-SiMS · QVHighlights val | mAP@avg: 31.5 -> 35.3 | Improved |
| use larger text encoder (E5-Mistral, NV-Embed) for query-caption similarity (ablation) | VTG-GPT · QVHighlights val | mAP@avg: 34.9 -> 30.87 / 31.44 | No clear effect |
| fuse frame and caption self-similarity maps versus frame-only (before re-ranking) (ablation) | Self-SiMS · QVHighlights val | mAP@avg: 38.2 -> 38.7 | Improved |
| swap LLaMA-3.2-Vision for MiniGPT-v2 in captioning and re-ranking (ablation) | Self-SiMS · QVHighlights val | mAP@avg: 39.3 -> 40.0 | No clear effect |
| add SMS to QMS on severe language-style gap subset (ablation) | Self-SiMS · QVHighlights val (severe LS-gap subset) | R1@0.5: 31.3 -> 31.7 | Improved |
| dynamic instance-wise threshold versus fixed thresholds (ablation) | Self-SiMS · three datasets | mIoU | Improved |
Closest papers by meaning
- Zero-shot Video Moment Retrieval via Off-the-shelf Multimodal Large Language Models (AAAI 2025 · similarity 0.93)
- Generative Video Diffusion for Unseen Novel Semantic Video Moment Retrieval (AAAI 2025 · similarity 0.89)
- Fewer Steps, Better Performance: Efficient Cross-Modal Clip Trimming for Video Moment Retrieval Using Language (AAAI 2024 · similarity 0.89)
- Towards Balanced Alignment: Modal-Enhanced Semantic Modeling for Video Moment Retrieval (AAAI 2024 · similarity 0.89)
- MomentSeeker: A Task-Oriented Benchmark For Long-Video Moment Retrieval (NeurIPS 2025 · similarity 0.88)
- When One Moment Isn't Enough: Multi-Moment Retrieval with Cross-Moment Interactions (NeurIPS 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
