Research map / Video / Video-language understanding
UMT: Unified Multi-Modal Transformers for Joint Video Moment Retrieval and Highlight Detection
Check your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add ASR-caption pretraining | UMT on QVHighlights · QVHighlights test | Avg. mAP: 36.12 -> 38.08 | Helped |
| Use video and audio together instead of video only (ablation) | UMT · YouTube Highlights | mAP: 73.48 -> 74.93 | Helped |
| Use video and audio together instead of audio only (ablation) | UMT · TVSum | Top-5 mAP: 76.51 -> 83.14 | Helped |
| Use video and audio together instead of video only (ablation) | UMT · QVHighlights val | MR mAP: 37.79 -> 38.59 | Helped |
| Train moment retrieval and highlight detection jointly instead of moment retrieval only (ablation) | UMT · QVHighlights val | MR mAP: 34.02 -> 38.59 | Helped |
| Train moment retrieval and highlight detection jointly instead of highlight detection only (ablation) | UMT · QVHighlights val | HD mAP: 40.22 -> 39.85 | Hurt |
| Add offset loss (ablation) | UMT moment retrieval · QVHighlights val | Avg. mAP: 37.36 -> 38.59 | Helped |
| Add a text query (ablation) | UMT highlight detection · TVSum | Top-5 mAP: 81.42 -> 83.14 | Helped |
| Add a text query (ablation) | UMT highlight detection · QVHighlights val | HD mAP: 25.14 -> 39.85 | Helped |
| Replace bottleneck fusion with concatenation (ablation) | UMT · YouTube Highlights | mAP: 74.93 -> 73.32 | Hurt |
| Replace bottleneck fusion with mean fusion (ablation) | UMT · TVSum | Top-5 mAP: 83.14 -> 81.76 | Hurt |
| Replace bottleneck fusion with sum fusion (ablation) | UMT · QVHighlights val | MR mAP: 38.59 -> 37.33 | Hurt |
Closest papers by meaning
- Bridging the Gap: A Unified Video Comprehension Framework for Moment Retrieval and Highlight Detection (CVPR 2024 · similarity 0.90)
- TR-DETR: Task-Reciprocal Transformer for Joint Moment Retrieval and Highlight Detection (AAAI 2024 · similarity 0.90)
- Task-Driven Exploration: Decoupling and Inter-Task Feedback for Joint Moment Retrieval and Highlight Detection (CVPR 2024 · similarity 0.88)
- Fewer Steps, Better Performance: Efficient Cross-Modal Clip Trimming for Video Moment Retrieval Using Language (AAAI 2024 · similarity 0.87)
- When One Moment Isn't Enough: Multi-Moment Retrieval with Cross-Moment Interactions (NeurIPS 2025 · similarity 0.87)
- Zero-shot Video Moment Retrieval via Off-the-shelf Multimodal Large Language Models (AAAI 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.