Research map / Video / Video-language understanding
Towards Video Thinking Test: A Holistic Benchmark for Advanced Video Reasoning and Understanding
Yuanhan Zhang, Yunice Chew, Yuhao Dong, Aria Leo, Bo Hu, Ziwei Liu
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use a larger model (72B instead of 7B) (ablation) | Qwen2.5-VL · Video-TT | robustness score: 14.4 -> 22.2 | Improved |
| use a larger model (72B instead of 7B) (ablation) | LLaVA-Video · Video-TT | robustness score: 13.7 -> 19.7 | Improved |
| scale InternVL-2.5 from 8B to 38B (ablation) | InternVL-2.5 · Video-TT | robustness score: 10.9 -> 11.1 | No clear effect |
| use video frames instead of language-only blind input (ablation) | GPT-4o · Video-TT | primary open-ended accuracy: 8.5 -> 36.6 | Improved |
| give correct cues in the question (correctly-led) instead of primary question | GPT-4o (blind) · Video-TT | accuracy: 8.5 -> 58.9 | Improved |
| add Chain-of-Thought prompt (Let's think step by step) (ablation) | video LLMs · Video-TT wrongly-led | relative accuracy change: about 6.8% relative increase | Improved |
| add Chain-of-Thought prompt (ablation) | video LLMs · Video-TT multiple-choice | accuracy | No clear effect |
| add audio transcript (ablation) | GPT-4o · Video-TT multiple-choice | accuracy | No clear effect |
| add audio transcript (ablation) | GPT-4o · Video-TT | robustness score: almost 15% relative gain | Improved |
| increase the number of input frames beyond 8 (ablation) | video LLMs · Video-TT | accuracy | No clear effect |
| increase the number of input frames up to 64 (ablation) | human annotators · Video-TT | accuracy | Improved |
| use multiple-choice instead of primary open-ended format | LLaVA-Video-72B · Video-TT | accuracy: 24.4 -> 47.5 | Improved |
Closest papers by meaning
- MMBench-Video: A Long-Form Multi-Shot Benchmark for Holistic Video Understanding (NeurIPS 2024 · similarity 0.91)
- MMR-V: What's Left Unsaid? A Benchmark for Multimodal Deep Reasoning in Videos (ICLR 2026 · similarity 0.90)
- HumanPCR: Probing MLLM Capabilities in Diverse Human-Centric Scenes (ICLR 2026 · similarity 0.89)
- VideoHallu: Evaluating and Mitigating Multi-modal Hallucinations on Synthetic Video Understanding (NeurIPS 2025 · similarity 0.88)
- VideoReasonBench: Can MLLMs Perform Vision-Centric Complex Video Reasoning? (ICLR 2026 · similarity 0.88)
- Thinking with Video: Video Generation as a Promising Multimodal Reasoning Paradigm (CVPR 2026 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
