Research map / Video / Video-language understanding
Detecting Moments and Highlights in Videos via Natural Language Queries
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add saliency loss (ablation) | Moment-DETR · QVHighlights val | highlight mAP: 17.84 -> 35.65 | Helped |
| add saliency loss (ablation) | Moment-DETR · QVHighlights val | R1@0.5: 44.84 -> 53.94 | Helped |
| add L1 loss or gIoU loss (each) (ablation) | Moment-DETR · QVHighlights val | moment retrieval | Helped |
| weakly supervised ASR pretraining | Moment-DETR · QVHighlights test | R1@0.5: 52.89 -> 59.78 | Helped |
| weakly supervised ASR pretraining | Moment-DETR · QVHighlights test | highlight mAP: 35.69 -> 37.43 | Helped |
| weakly supervised ASR pretraining | Moment-DETR · QVHighlights test | mAP@0.75: 29.40 -> 35.36 | Helped |
| out-of-domain HowTo100M pretraining, 2.5K to 10K videos (ablation) | Moment-DETR · QVHighlights val | R1@0.5: 53.94 -> 56.06 | Helped |
| out-of-domain HowTo100M pretraining (ablation) | Moment-DETR · QVHighlights val | highlight mAP: 35.65 -> 35.97 | Mixed |
| in-domain 5.4K QVHighlights pretraining instead of 10K HowTo100M (ablation) | Moment-DETR · QVHighlights val | R1@0.5: 56.06 -> 59.68 | Helped |
| Moment-DETR vs improved XML+ (no human priors) | Moment-DETR · QVHighlights test | moment retrieval: R1@0.5 46.69 -> 52.89; R1@0.7 33.46 -> 33.02 | Mixed |
| pretrain on 10K HowTo100M | Moment-DETR · CharadesSTA test | R1@0.5: 53.63 -> 55.65 | Helped |
| increase XML capacity and add saliency loss (XML+) | XML · QVHighlights test | avg mAP: 32.14 -> 34.90 | Helped |
Closest papers by meaning
- TR-DETR: Task-Reciprocal Transformer for Joint Moment Retrieval and Highlight Detection (AAAI 2024 · similarity 0.89)
- Fewer Steps, Better Performance: Efficient Cross-Modal Clip Trimming for Video Moment Retrieval Using Language (AAAI 2024 · similarity 0.89)
- Generative Video Diffusion for Unseen Novel Semantic Video Moment Retrieval (AAAI 2025 · similarity 0.88)
- Zero-shot Video Moment Retrieval via Off-the-shelf Multimodal Large Language Models (AAAI 2025 · similarity 0.87)
- Watch Video, Catch Keyword: Context-aware Keyword Attention for Moment Retrieval and Highlight Detection (AAAI 2025 · similarity 0.87)
- Bridging the Gap: A Unified Video Comprehension Framework for Moment Retrieval and Highlight Detection (CVPR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.