Research map / Video / Video-language understanding
MUVR: A Multi-Modal Untrimmed Video Retrieval Benchmark with Multi-Level Visual Correspondence
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces MUVR, a benchmark for multi-modal untrimmed video retrieval on Bilibili videos with video+text+tag+mask queries, multi-level visual correspondence, and a Reranking Score for evaluating MLLMs.
Problem
Video retrieval for long-video platforms lacks benchmarks that support untrimmed videos, one-to-many retrieval, and video-centric multi-modal queries; MLLM reranking for video retrieval is also under-explored.
Approach
Video-centric multi-modal queries (video + long text description + tag prompt + mask prompt) matched against untrimmed videos under six levels of visual correspondence (copy, event, scene, instance, action, others), plus a Reranking Score that rewards keeping true matches and removing false ones when MLLMs judge query-target pairs.
Key results
- EVA-CLIP with multimodal query reaches 58.0% mAP on MUVR-Base, while adding tag prompts on MUVR-Filter yields only 34.0% mAP for EVA-CLIP.
- On MUVR-QA, GPT-4o with 6 frames and multi-image comparison reaches 65.0% accuracy and a 0.19 Reranking Score (no-tag subset: 75.5%).
- S2VS improves over its CLIP (RN50x4) base from 34.2% to 47.2% mAP under pure video queries on MUVR-Base.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Combine video and text queries instead of using video alone | EVA-CLIP on MUVR-Base · MUVR-Base | mAP: 50.7 -> 58.0 | Helped |
| Combine video and text queries instead of using text alone | EVA-CLIP on MUVR-Base · MUVR-Base | mAP: 43.0 -> 58.0 | Helped |
| Use S2VS retrieval framework with CLIP RN50x4 features | Video-query retrieval on MUVR-Base · MUVR-Base | mAP: 34.2% to 47.2% | Helped |
| Add a tag prompt to the multimodal query (ablation) | EVA-CLIP on MUVR-Filter · MUVR-Filter | mAP: 32.9 -> 34.0 | Helped |
| Add a tag prompt to the multimodal query (ablation) | BLIP on MUVR-Filter · MUVR-Filter | mAP: 25.5 -> 25.6 | No clear effect |
| Add a tag prompt to the multimodal query (ablation) | OpenCLIP ViT-H-14 on MUVR-Filter · MUVR-Filter | mAP: 30.8 -> 31.7 | Helped |
| Increase multi-image input from 1 to 6 frames (ablation) | InternVL2.5 on MUVR-QA · MUVR-QA | Accuracy: 52.0% to 57.0% | Helped |
| Increase multi-image input from 1 to 6 frames (ablation) | InternVL2.5 on MUVR-QA · MUVR-QA | Reranking Score: -0.66 -> -0.35 | Helped |
| Increase multi-image input from 6 to 12 frames (ablation) | InternVL2.5 on MUVR-QA · MUVR-QA | Accuracy: 57.0 -> 56.5 | Mixed |
| Add a mask prompt (ablation) | VideoRefer on MUVR-QA · MUVR-QA | Reranking Score: 0.05 -> 0.07 | Helped |
| Jointly compare query and target video frames instead of using text query-only comparison | InternVL2 on MUVR-QA · MUVR-QA | Reranking Score: -0.52 -> -0.23 | Helped |
| Use multimodal queries for the Instance partition rather than video-only queries | CLIP RN50x4 on MUVR-Base · MUVR-Base Instance | mAP: 31.8% vs. 41.0% | Helped |
Surprising findings
- Image-based VLMs outperform video models for pure text queries, attributed to user-generated image-text pairs in training data.
- Adding tag prompts gives only marginal gains for embedding models, and tag prompts substantially reduce MLLM accuracy on MUVR-QA.
Limitations
- Relies on human annotation, which may contain minor errors.
- Focuses only on visual and textual modalities, excluding audio.
- Tag prompts limited to single positive or negative tags in evaluation.
Experimental setup
Datasets: MUVR-Base MUVR-Filter MUVR-QA
Benchmarks: MUVR-Base MUVR-Filter MUVR-QA
Metrics: mAP uAP Recall@k Accuracy Reranking Score
Baselines: CLIP OpenCLIP EVA-CLIP BLIP BLIP2 InternVideo2 S2VS CoVR InternVL2 InternVL2.5 MiniCPM-o 2.6 MiniCPM-V 2.6
Builds on: FIVR CoVR SAM2 BLIP-2
Closest papers by meaning
- From Play to Replay: Composed Video Retrieval for Temporally Fine-Grained Videos (NeurIPS 2025 · similarity 0.89)
- CoVR: Learning Composed Video Retrieval from Web Video Captions (AAAI 2024 · similarity 0.88)
- Bridging the Semantic Granularity Gap Between Text and Frame Representations for Partially Relevant Video Retrieval (AAAI 2025 · similarity 0.88)
- Video-RAG: Visually-aligned Retrieval-Augmented Long Video Comprehension (NeurIPS 2025 · similarity 0.88)
- Composed Video Retrieval via Enriched Context and Discriminative Embeddings (CVPR 2024 · similarity 0.87)
- Referred by Multi-Modality: A Unified Temporal Transformer for Video Object Segmentation (AAAI 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.