Research map / Video / Video-language understanding
Adapting MLLMs for Nuanced Video Retrieval
Piyush Nitin Bagad, Andrew Zisserman
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| fine-tune MLLM on 20K text-only triplets with nuance hard negatives (TARA) | Tarsier 2 · CiA SSv2 (Chiral) | mAP: 77.7 -> 88.9 | Improved |
| apply TARA to autoregressive base MLLMs | Qwen2VL / InternVL3 · CiA SSv2 (Chiral) | mAP: 56.8 -> 73.1 (InternVL3); 60.2 -> 70.1 (Qwen2VL) | Improved |
| apply TARA to already contrastively trained Qwen3VL-Embedding | Qwen3VL-Embedding · CiA SSv2 (Chiral) | mAP: 72.0 -> 73.0 | Improved |
| TARA on Qwen3VL-Embedding for EPIC static | Qwen3VL-Embedding · CiA EPIC (Static) | mAP: 28.6 -> 25.1 | Hurt |
| text-only TARA fine-tuning vs multimodal fine-tuning on same Qwen2VL base | Qwen2VL-7B · Nuanced retrieval average | avg score: 51.9 -> 56.8 | Improved |
| TARA zero-shot on negation queries | Tarsier 2 · NegBench COCO | R-Neg@5: 21.5 -> 73.6 | Improved |
| TARA zero-shot on composed video retrieval | Tarsier 2 · WebVid-CoVR | R@1: 59.8 -> 66.3 | Improved |
| TARA on standard video benchmarks | Tarsier 2 · MMEB-V2 video classification | Avg.: 27.4 -> 66.1 | Improved |
| TARA vs Qwen3-VL-Embedding on standard benchmarks | Tarsier 2 + TARA · MMEB-V2 video retrieval | Avg.: 45.9 vs 58.9 | Hurt |
| concatenate TARA and Qwen3VL-Embedding embeddings | Qwen3VL-Embedding · MMEB-V2 video retrieval | Avg.: 58.9 -> 59.5 | Improved |
| increase training data size from 5K to 200K text triplets (ablation) | TARA · CiA SSv2 (Chiral) | mAP | No clear effect |
| use EOL prompt alone to close modality gap (no fine-tuning) (ablation) | Qwen2VL · MSRVTT video-text | modality gap | No clear effect |
Closest papers by meaning
- NUDGE: Lightweight Non-Parametric Fine-Tuning of Embeddings for Retrieval (ICLR 2025 · similarity 0.90)
- U-MARVEL: Unveiling Key Factors for Universal Multimodal Retrieval via Embedding Learning with MLLMs (ICLR 2026 · similarity 0.90)
- MM-EMBED: UNIVERSAL MULTIMODAL RETRIEVAL WITH MULTIMODAL LLMS (ICLR 2025 · similarity 0.89)
- MetaEmbed: Scaling Multimodal Retrieval at Test-Time with Flexible Late Interaction (ICLR 2026 · similarity 0.89)
- Mitigating Semantic Collapse in Partially Relevant Video Retrieval (NeurIPS 2025 · similarity 0.88)
- Your Embedding Model Is SMARTer Than You Think (NeurIPS 2026 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
