Research map / Video / Video-language understanding
Revisiting the "Video" in Video-Language Understanding
Check your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Select one frame embedding from a sampled set rather than use a single-frame CLIP baseline | CLIP ViT-B-32 on MSR-VTT-MC · MSR-VTT-MC | accuracy: 84.8 -> 93.2 | Helped |
| Increase the sampled frame set from 4 to 8 for ATP selection (ablation) | ATP on MSR-VTT-MC · MSR-VTT-MC | accuracy: 91.4 -> 92.5 | Helped |
| Increase the sampled frame set from 8 to 16 for ATP selection (ablation) | ATP on MSR-VTT-MC · MSR-VTT-MC | accuracy: 92.5 -> 93.2 | Helped |
| Use ATP frame selection instead of mean-pooled CLIP frame features | CLIP ViT-B-32 on VALUE-How2QA · VALUE-How2QA | accuracy: 55.7 -> 65.1 | Helped |
| Use ATP frame selection instead of a single-frame CLIP baseline | CLIP ViT-B-32 on NExT-QA · NExT-QA (main dataset) | accuracy: 43.7 -> 49.2 | Helped |
| Add temporal reasoning over ATP-selected frame embeddings | ATP on NExT-QA · NExT-QA (main dataset) | accuracy: 49.2 -> 51.5 | Helped |
| Use ATP confidence to route easy questions to ATP and harder ones to the temporal model | Temp[ATP] on NExT-QA · NExT-QA (main dataset) | accuracy: 51.5 -> 54.3 | Helped |
| Use ATP-selected candidates with temporal reasoning on the ATP-hard subset | ATP on NExT-QA ATP-hard subset · NExT-QA (ATP-hard subset) | accuracy: 20.2 -> 38.8 | Helped |
| Use ATP selection instead of single-frame CLIP features for text-to-video retrieval | CLIP ViT-B-32 on MSR-VTT · MSR-VTT retrieval | R@1: 21.6 -> 27.8 | Helped |
| Use ATP selection instead of single-frame CLIP features for text-to-video retrieval | CLIP ViT-B-32 on DiDeMo · DiDeMo retrieval | R@1: 20.2 -> 26.1 | Helped |
| Use ATP selection instead of single-frame CLIP features for text-to-video retrieval | CLIP ViT-B-32 on ActivityNet · ActivityNet retrieval | R@1: 12.5 -> 17.7 | Helped |
Closest papers by meaning
- E.T. Bench: Towards Open-Ended Event-Level Video-Language Understanding (NeurIPS 2024 · similarity 0.92)
- LongVideoBench: A Benchmark for Long-context Interleaved Video-Language Understanding (NeurIPS 2024 · similarity 0.90)
- HourVideo: 1-Hour Video-Language Understanding (NeurIPS 2024 · similarity 0.90)
- Needle In A Video Haystack: A Scalable Synthetic Evaluator for Video MLLMs (ICLR 2025 · similarity 0.90)
- SF2T: Self-supervised Fragment Finetuning of Video-LLMs for Fine-Grained Understanding (CVPR 2025 · similarity 0.90)
- MMBench-Video: A Long-Form Multi-Shot Benchmark for Holistic Video Understanding (NeurIPS 2024 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.