Research map / Video / Video-language understanding
When One Moment Isn't Enough: Multi-Moment Retrieval with Cross-Moment Interactions
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces QV-M² (a manually annotated multi-moment retrieval dataset built on QVHighlights) with multi-moment metrics, and FlashMMR, which adds a post-verification module on top of FlashVTG-style moment retrieval.
Problem
Existing moment retrieval benchmarks and methods assume a single relevant moment per query, which does not match real-world queries that correspond to multiple moments in a video.
Approach
Extends QVHighlights videos with multi-moment human annotations (QV-M²) and proposes FlashMMR, which adds a post-verification module that re-scores predicted moments with a GRU over moment features, supervised by tIoU, plus a self-similarity representation loss.
Key results
- FlashMMR reaches 35.14 G-mAP on the QV-M² test set, versus 32.14 for FlashVTG.
- On QVHighlights validation, FlashMMR reaches 48.07 G-mAP, marginally above FlashVTG's 48.02.
- Models trained on QV-M² and tested on QVHighlights (FlashMMR) reach 48.42 G-mAP, the highest in the cross-dataset table.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add post-verification module (ablation) | FlashMMR · QV-M² test | G-mAP: 32.14 -> 35.14 | Helped |
| add post-verification module (ablation) | FlashMMR · QV-M² test | mAP@2_tgt: 39.48 -> 42.52 | Helped |
| add post-verification module (ablation) | FlashMMR · QV-M² test | mAP@3+tgt: 20.19 -> 22.89 | Helped |
| add post-verification module (ablation) | FlashMMR · QV-M² test | mIoU@2: 47.85 -> 49.64 | Helped |
| add post-verification module (ablation) | FlashMMR · QV-M² test | mR@2: 41.30 -> 44.33 | Helped |
| add post-verification module (ablation) | FlashMMR · QVHighlights validation | G-mAP: 48.02 -> 48.07 | Helped |
| add post-verification module (ablation) | FlashMMR · QVHighlights validation | mAP@2_tgt: 35.08 -> 35.78 | Helped |
| add post-verification module (ablation) | FlashMMR · QVHighlights validation | mIoU@2: 43.80 -> 45.32 | Helped |
Surprising findings
- Existing SMR models show a performance drop when evaluated on QV-M², indicating they struggle with one-to-many queries.
- On QVHighlights, the post-verification module gives only small gains (e.g., 0.05 G-mAP), with FlashVTG nearly matching FlashMMR's G-mAP.
Limitations
- The post-verification module is described by the authors as being in an early stage.
- The QV-M² dataset is relatively small for training advanced models.
- No error bars or statistical significance tests are reported.
Experimental setup
Datasets: QVHighlights QV-M² Charades-STA
Benchmarks: QV-M² QVHighlights
Metrics: G-mAP mAP@1_tgt mAP@2_tgt mAP@3+_tgt mIoU@k mR@k
Baselines: M-DETR EATR QD-DETR TR-DETR CG-DETR FlashVTG UVCOM
Builds on: FlashVTG QVHighlights Moment-DETR QD-DETR EATR
Closest papers by meaning
- MomentSeeker: A Task-Oriented Benchmark For Long-Video Moment Retrieval (NeurIPS 2025 · similarity 0.89)
- Fewer Steps, Better Performance: Efficient Cross-Modal Clip Trimming for Video Moment Retrieval Using Language (AAAI 2024 · similarity 0.89)
- Zero-shot Video Moment Retrieval via Off-the-shelf Multimodal Large Language Models (AAAI 2025 · similarity 0.88)
- Generative Video Diffusion for Unseen Novel Semantic Video Moment Retrieval (AAAI 2025 · similarity 0.88)
- Transferable Video Moment Localization by Moment-Guided Query Prompting (AAAI 2024 · similarity 0.87)
- VERIFIED: A Video Corpus Moment Retrieval Benchmark for Fine-Grained Video Understanding (NeurIPS 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.