Research map / Vision-language and multimodal / Visual question answering
MovieRecapsQA: A Multimodal Open-Ended Video Question-Answering Benchmark
Shaden Shaar, Bradon Thymes, Sirawut Chaixanien, Claire Cardie, Bharath Hariharan
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| remove visual input (dialogue only) (ablation) | proprietary MLLMs · MovieRecapsQA vision-centric | Factuality: +0.73 | Improved |
| remove visual input (dialogue only) (ablation) | proprietary MLLMs · MovieRecapsQA dialogue-centric | Factuality: +0.49 | Improved |
| remove visual input (dialogue only) (ablation) | open-source MLLMs · MovieRecapsQA | Factuality: -0.01 to +0.10 on question types | No clear effect |
| use frames only (remove dialogue) (ablation) | proprietary MLLMs · MovieRecapsQA dialogue-centric | Factuality: -0.48 | Hurt |
| use frames only (remove dialogue) (ablation) | proprietary MLLMs · MovieRecapsQA setting/technical analysis | Factuality: -0.19 | Hurt |
| use frames only (remove dialogue) (ablation) | open-source MLLMs · MovieRecapsQA | Factuality: -0.14 (dialogue) to +0.12 (STA) | Mixed |
| reference-free fact-grounded metric vs semantic metrics | evaluation · MovieRecapsQA | model separation: Factuality 2.96-3.99 vs ROUGE-L 0.22-0.28 | Improved |
| reference-based HELMET Correctness vs humans | evaluation · MovieRecapsQA | HELMET Correctness: MiniCPM-o 1.27 vs avg human 0.98 | Hurt |
Closest papers by meaning
- LongVideoBench: A Benchmark for Long-context Interleaved Video-Language Understanding (NeurIPS 2024 · similarity 0.91)
- CG-Bench: Clue-grounded Question Answering Benchmark for Long Video Understanding (ICLR 2025 · similarity 0.89)
- Q-Bench-Video: Benchmark the Video Quality Understanding of LMMs (CVPR 2025 · similarity 0.89)
- MMOU: A Massive Multi-Task Omni Understanding and Reasoning Benchmark for Long and Complex Real-World Videos (NeurIPS 2026 · similarity 0.89)
- MAGNET: A Multi-agent Framework for Finding Audio-Visual Needles by Reasoning over Multi-Video Haystacks (NeurIPS 2025 · similarity 0.89)
- MMR-V: What's Left Unsaid? A Benchmark for Multimodal Deep Reasoning in Videos (ICLR 2026 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
