Research map / Video / Video-language understanding
Transferable Video Moment Localization by Moment-Guided Query Prompting
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Studies cross-domain transferability of video moment localization and proposes MGQP, which turns query sentences into protagonist/action/object prompt primitives refined with visual guidance and global prompts on top of frozen pre-trained vision-language models.
Problem
Video moment localization models are typically trained and tested within a single domain, so their transferability across domains with large visual and textual gaps is unexplored; query sentences also differ in style from text used to pre-train vision-language models.
Approach
Decompose queries into protagonist, action, and object prompt primitives with learnable context, refine them with visual-moment and query guidance, and add learnable global prompts processed by a light transformer, all while keeping the pre-trained vision-language encoders frozen.
Key results
- On TACoS to Charades, MGQP reaches 39.36 Rank@1 IoU=0.3 versus 36.53 for CoOp and 36.21 for MMN.
- On TACoS to YouCookII, MGQP reaches 9.20 Rank@1 IoU=0.3 versus 8.13 for CoOp.
- On Charades to TACoS, MGQP reaches 10.24 Rank@1 IoU=0.3 versus 7.90 for CoOp on the same IoU threshold.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add prompt primitives that separate protagonist, action, and object information (ablation) | MGQP · TACoS → Charades | Rank@5, IoU=0.5: 69.82 -> 71.30 | Helped |
| Add the multimodal prompt refiner (ablation) | MGQP · TACoS → Charades | Rank@5, IoU=0.5: 69.41 -> 71.30 | Helped |
| Add the holistic prompt incorporator (ablation) | MGQP · TACoS → Charades | Rank@5, IoU=0.5: 71.08 -> 71.30 | Helped |
| Use MGQP instead of the strongest listed baseline | video moment localization with ActionCLIP · TACoS → Charades | Rank@1, IoU=0.3: 36.53 -> 39.36 | Helped |
| Use MGQP instead of the strongest listed baseline | video moment localization with ActionCLIP · TACoS → YouCookII | Rank@1, IoU=0.3: 8.13 -> 9.20 | Helped |
| Use MGQP instead of the strongest listed baseline | video moment localization with ActionCLIP · Charades → TACoS | Rank@1, IoU=0.3: 28.90 -> 29.56 | Helped |
| Use MGQP instead of the strongest listed baseline | video moment localization with ActionCLIP · DiDeMo → YouCookII | Rank@1, IoU=0.3: 7.94 -> 8.51 | Helped |
Surprising findings
- Simply swapping in pre-trained vision-language features (MMN-Feature) yields no substantial gain over the original MMN baseline.
- CoCoOp has the highest Rank@5 IoU=0.3 on TACoS to Charades (89.03) among compared methods, but lower Rank@1 than CoOp.
Limitations
- Prompt primitives are relatively simple and struggle with queries containing complex relationships, as shown in the failure case.
- Rule-based chunking parsers are used to populate primitives, with empirically chosen rules.
- Evaluated on only a small set of cross-domain pairs and four datasets.
Experimental setup
Datasets: TACoS Charades-STA DiDeMo YouCookII MSVD MSR-VTT WebVid
Benchmarks: TACoS to Charades-STA TACoS to YouCookII Charades-STA to TACoS DiDeMo to YouCookII
Metrics: Rank@1 IoU=0.3/0.5/0.7 Rank@5 IoU=0.3/0.5/0.7
Baselines: 2D-TAN VSLNet EAMAT MMN MMN-Feature CoOp CoCoOp
Builds on: CoOp CoCoOp MMN ActionCLIP CLIP4Clip
Closest papers by meaning
- Fewer Steps, Better Performance: Efficient Cross-Modal Clip Trimming for Video Moment Retrieval Using Language (AAAI 2024 · similarity 0.89)
- Generative Video Diffusion for Unseen Novel Semantic Video Moment Retrieval (AAAI 2025 · similarity 0.88)
- MomentSeeker: A Task-Oriented Benchmark For Long-Video Moment Retrieval (NeurIPS 2025 · similarity 0.88)
- Boundary-Aware Temporal Dynamic Pseudo-Supervision Pairs Generation for Zero-Shot Natural Language Video Localization (AAAI 2025 · similarity 0.88)
- Localizing Events in Videos with Multimodal Queries (CVPR 2025 · similarity 0.88)
- Zero-shot Video Moment Retrieval via Off-the-shelf Multimodal Large Language Models (AAAI 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.