Research map / Video / Video-language understanding
OmniCVR: A Benchmark for Omni-Composed Video Retrieval with Vision, Audio, and Text
Junyang Ji, Shengjun Zhang, Da Li, Yuxiao Luo, Yan Wang, Di Xu, Biao Yang, Wei Yuan, Fan Yang, Zhihai He, Wenming Yang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add audio-to-text descriptions from Qwen2-Audio | VLM2Vec · OmniCVR audio-centric | R@1: 12.4 -> 77.2 | Improved |
| add audio-to-text descriptions from Qwen2-Audio | VLM2Vec · OmniCVR (overall) | R@1: 38.44 -> 66.98 | Improved |
| add audio-to-text descriptions from Qwen2-Audio | VLM2Vec · OmniCVR (vision-centric) | R@1: 55.04 -> 69.33 | Improved |
| add audio-to-text descriptions, human speech targets | VLM2Vec · OmniCVR audio-centric | R@1: 11.36 -> 96.59 | Improved |
| add audio-to-text descriptions, music targets | VLM2Vec · OmniCVR audio-centric | R@1: 16.07 -> 86.43 | Improved |
| add audio-to-text descriptions, sound targets | VLM2Vec · OmniCVR audio-centric | R@1: 10.75 -> 60.31 | Improved |
| add source video frames as input (ablation) | AudioVLM2Vec · OmniCVR audio-centric | R@1: 28.10 -> 77.20 | Improved |
| replace native audio tokens with Audio-as-Text captions (ablation) | OmniEmbed-v0.1-multivent · OmniCVR audio-centric | R@1: 13.6 -> 32.7 | Improved |
| replace native audio tokens with Audio-as-Text captions, speech (ablation) | OmniEmbed-v0.1-multivent · OmniCVR audio-centric | R@1: 11.36 -> 50.38 | Improved |
| replace native audio tokens with Audio-as-Text captions, music (ablation) | OmniEmbed-v0.1-multivent · OmniCVR audio-centric | R@1: 16.07 -> 30.00 | Improved |
| add audio-to-text descriptions zero-shot | VLM2Vec · MSR-VTT zero-shot T2V | R@1: 36.10 -> 37.90 | Improved |
| add audio captioning step (latency cost) | VLM2Vec · OmniCVR audio-centric | inference latency (s): 1.72 -> 4.77 | Hurt |
Closest papers by meaning
- OmniVideoBench: Towards Audio-Visual Understanding Evaluation for Omni MLLMs (ICLR 2026 · similarity 0.91)
- JointAVBench: A Benchmark for Joint Audio-Visual Reasoning Evaluation (ICLR 2026 · similarity 0.91)
- Omni-Interactive Universal Embedder (NeurIPS 2026 · similarity 0.90)
- A Benchmark for Omni-Modal Reasoning in Long Videos (NeurIPS 2026 · similarity 0.90)
- AVOC: Enhancing Hour-Level Audio-Video Understanding in Omni-Modal LLMs via Retrieval-Inspired Token Compression (NeurIPS 2026 · similarity 0.89)
- MMOU: A Massive Multi-Task Omni Understanding and Reasoning Benchmark for Long and Complex Real-World Videos (NeurIPS 2026 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
