Research map / Video / Video-language understanding
MINERVA: Evaluating Complex Video Reasoning
Arsha Nagrani, Sachit Menon, Ahmet Iscen, Shyamal Buch, Ramin Mehran, Nilpa Jha, Anja Hauth, Yukun Zhu, Carl Vondrick, Mikhail Sirotenko, Cordelia Schmid, Tobias Weyand
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add ASR transcript to frames (ablation) | Gemini 2.0 Flash (256 frames) · MINERVA | MCQ accuracy: 49.70 -> 53.47 | Improved |
| add ASR transcript to frames (ablation) | GPT-4o (250 frames) · MINERVA | MCQ accuracy: 44.82 -> 45.54 | Improved |
| add ASR transcript to frames (ablation) | Qwen-2.5VL (768 frames) · MINERVA | MCQ accuracy: 34.39 -> 35.05 | Improved |
| increase frames from 64 to 256 (ablation) | Gemini 2.0 Flash · MINERVA | MCQ accuracy: 47.85 -> 53.47 | Improved |
| increase frames from 64 to 250 (ablation) | GPT-4o · MINERVA | MCQ accuracy: 39.74 -> 45.54 | Improved |
| increase frames from 64 to 768 (ablation) | Qwen-2.5VL · MINERVA | MCQ accuracy: 33.00 -> 35.05 | Improved |
| add step-by-step reasoning prompt (ablation) | Gemini 2.0 Flash · MINERVA | MCQ accuracy: 46.47 -> 51.22 | Improved |
| add MINERVA rubric to prompt (ablation) | Gemini 2.0 Flash · MINERVA | MCQ accuracy: 51.22 -> 53.47 | Improved |
| add MINERVA rubric to prompt (ablation) | Gemini 2.0 Flash · MINERVA | MiRA: 0.65 -> 0.75 | Improved |
| turn on thinking mode (ablation) | Gemini 2.5 Pro (1024 frames) · MINERVA | MCQ accuracy: 63.9 -> 66.2 | Improved |
| turn on thinking mode (ablation) | Gemini 2.5 Pro (256 frames) · MINERVA | MCQ accuracy: 63.1 -> 64.7 | Improved |
| turn on thinking mode (ablation) | Gemini 2.5 Pro (512 frames) · MINERVA | MCQ accuracy: 62.3 -> 66.0 | Improved |
Closest papers by meaning
- SciVideoBench: Benchmarking Scientific Video Reasoning in Large Multimodal Models (ICML 2026 · similarity 0.91)
- MMR-V: What's Left Unsaid? A Benchmark for Multimodal Deep Reasoning in Videos (ICLR 2026 · similarity 0.91)
- TOMATO: Assessing Visual Temporal Reasoning Capabilities in Multimodal Foundation Models (ICLR 2025 · similarity 0.89)
- VideoAutoArena: An Automated Arena for Evaluating Large Multimodal Models in Video Analysis through User Simulation (CVPR 2025 · similarity 0.89)
- RPTS: Tree-Structured Reasoning Process Scoring for Faithful Multimodal Evaluation (AAAI 2026 · similarity 0.89)
- VideoReasonBench: Can MLLMs Perform Vision-Centric Complex Video Reasoning? (ICLR 2026 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
