Research map / Vision-language and multimodal / Multimodal LLMs

VideoReasonBench: Can MLLMs Perform Vision-Centric Complex Video Reasoning?

ICLR 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Chain-of-thought multimodal reasoning · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces VideoReasonBench, a synthetic-and-real video benchmark of 1,440 questions over 240 videos that tests vision-centric multi-step state tracking, finding most MLLMs score under 10% while Gemini-2.5-Pro reaches 56%.

Problem

Existing video QA benchmarks lack the reasoning depth and visual reliance needed to show the benefit of long chain-of-thought reasoning; recent video reasoning benchmarks are largely knowledge-driven.

Approach

Videos show a sequence of fine-grained operations on a latent state that is visible only at the start or end, so models must recall operations, infer hidden states, and predict beyond the video across three escalating levels (recall, infer, predict).

Key results

• Gemini-2.5-Pro-0506 reaches 56.0% overall accuracy on VideoReasonBench versus 73.8% for humans, while GPT-4o gets 6.9%.
• Gemini-2.5-Flash drops from 27.4% to 12.2% when half the video is cut, and to 0.5% with a single frame, whereas existing benchmarks drop at most 28.3%.
• Raising Gemini-2.5-Flash's thinking budget improves VideoReasonBench by roughly 9 points while existing benchmarks gain under 2.5 points.

Surprising findings

• Replacing video with a textual summary (vid2txt) raises Gemini-2.5-Flash from 27.4% to 72.2%, showing fine-grained temporal perception is the bottleneck.
• Most open-source MLLMs, including 72B-class models, score under 10%, with efficient models near zero.

Limitations

• Videos are controlled, synthetic-like scenarios (six demonstration types) that may not reflect real-world video reasoning.
• Evaluation relies on an LLM judge and rule-based state simulation, with only small paraphrase and judge-swap robustness checks.

Experimental setup

Benchmarks: VideoReasonBench TempCompass Video-MME MMVU Video-MMMU

Metrics: accuracy

Baselines: GPT-4o o4-mini Seed1.5-VL Gemini-2.0-Flash Gemini-2.5-Flash Gemini-2.5-Pro Qwen2.5-VL InternVL3 LLaVA-OneVision LLaVA-Video MiniCPM-V 2.6 MiniCPM-o 2.6

Builds on: Video-MME TempCompass Video-MMMU MMVU VLMEvalKit

Closest papers by meaning

• MMBench-Video: A Long-Form Multi-Shot Benchmark for Holistic Video Understanding (NeurIPS 2024 · similarity 0.92)
• MVBench: A Comprehensive Multi-modal Video Understanding Benchmark (CVPR 2024 · similarity 0.91)
• VideoRFT: Incentivizing Video Reasoning Capability in MLLMs via Reinforced Fine-Tuning (NeurIPS 2025 · similarity 0.91)
• Incentivizing Versatile Video Reasoning in MLLMs via Data-Efficient Reinforcement Learning (CVPR 2026 · similarity 0.91)
• SciVideoBench: Benchmarking Scientific Video Reasoning in Large Multimodal Models (ICML 2026 · similarity 0.91)
• Video-R1: Reinforcing Video Reasoning in MLLMs (NeurIPS 2025 · similarity 0.90)