Research map / Video / Video-language understanding
Rethinking Video-Text Understanding: Retrieval from Counterfactually Augmented Data
Wufei Ma, Kai Li, Zhongshi Jiang, Moustafa Meshry, Qihao Liu, Huiyu Wang, Christian Haene, Alan Yuille
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add LLM-teacher with soft logits to InternVideo | InternVideo · Feint6K (MSR-VTT) | R@1: 58.6 -> 65.8 | Improved |
| add LLM-teacher with soft logits to SimVTP | SimVTP · Feint6K (MSR-VTT) | R@1: 35.7 -> 43.5 | Improved |
| use binary pseudo-labels instead of soft logits (ablation) | LLM-teacher on InternVideo · VATEX retrieval | R@1: 87.3 -> 85.2 | Hurt |
| add LLM-teacher soft logits | InternVideo · VATEX standard retrieval | R@1: 87.9 -> 87.3 | No clear effect |
| use 5 instead of 10 action captions (ablation) | LLM-teacher · Feint6K (VATEX) | R@1: 65.6 -> 64.7 | Hurt |
| add object-based captions alongside action captions (ablation) | LLM-teacher · Feint6K (VATEX) | R@1: 65.6 -> 64.2 | Hurt |
| generate captions with LLM chatbot instead of mask filling (ablation) | LLM-teacher · Feint6K (VATEX) | R@1: 65.6 -> 65.9 | No clear effect |
| counterfactual negatives instead of random negatives (benchmark design) | InternVideo finetuned · MSR-VTT vs Feint6K | R@1: 49.1 -> 58.6 vs human 95.2 | Hurt |
Closest papers by meaning
- Leveraging Textual Memory and Key Frame Reasoning for Full Video Understanding Using Off-the-Shelf LLMs and VLMs (Student Abstract) (AAAI 2025 · similarity 0.88)
- TOPA: Extending Large Language Models for Video Understanding via Text-Only Pre-Alignment (NeurIPS 2024 · similarity 0.88)
- InternVid: A Large-scale Video-Text Dataset for Multimodal Understanding and Generation (ICLR 2024 · similarity 0.87)
- OST: Refining Text Knowledge with Optimal Spatio-Temporal Descriptor for General Video Recognition (CVPR 2024 · similarity 0.87)
- Not Another Text Benchmark: Putting the “Visual" Back in Visual Question Answering for Large Video Models (NeurIPS 2026 · similarity 0.87)
- Needle In A Video Haystack: A Scalable Synthetic Evaluator for Video MLLMs (ICLR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
