Research map / Video / Video-language understanding

AgenticVBench: Can AI Agents Complete Real-World Video Production Tasks?

NeurIPS 2026 · Video › Video-language understanding · approach: Event and anomaly reasoning in video

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces AgenticVBench, a 100-task expert-authored benchmark for agentic video post-production (Assembly, Repair, Sequencing, Repurpose), evaluating frontier VLMs across native and open-source harnesses and finding the best stack reaches about 30-31% versus expert human performance.

Problem

Existing multimodal benchmarks test single-shot perception/QA and existing agentic benchmarks focus on software engineering or browsers, so there is no measure of whether AI agents can complete composite, long-horizon real-world video production tasks.

Approach

A benchmark of 100 expert-authored video post-production tasks across four families, graded by programmatic verifiers and binary expert rubrics, with a systematic model-by-harness evaluation showing harness choice is a first-order factor in agent performance.

Key results

• The best model-harness combination scores about 31% overall (best Assembly 0.38, best Repair 0.30, best Repurpose 0.30), well below expert human reference (e.g., Repurpose 0.95, Assembly 0.81).
• Harness choice shifts GPT-5.5's Assembly score by 20 pp across Codex, OpenCode and OpenClaw (0.38 / 0.37 / 0.18).
• Qwen3-VL-235B on OpenCode scores 0.009 on Assembly versus 0.073 on OpenClaw under the same model.

Surprising findings

• Stripping cinematic shorthand fields (camera movement) changes Assembly score by about 1 pp, suggesting agents rely on prose descriptions rather than cinematic detail.
• A Repurpose agent produced a 60-second deliverable at $0.20 without ever reading source frames, showing the deliverable can satisfy structural requirements without ingesting source visuals.

Limitations

• Coverage limited to frontier models and a small set of harnesses; many models and frameworks not evaluated.
• Some rubric items are subjective and rely on human or VLM grading; the calibrated grader's details are sparse.
• Tasks emphasize post-production from a limited expert pool and public source videos; limited languages, regions and genres.

Experimental setup

Datasets: Runway AI Film Festival 2025 submissions publicly accessible source videos

Benchmarks: AgenticVBench

Metrics: normalized task score per-family score human expert reference gap tool-call count tool concentration failure reason share

Baselines: Claude Opus 4.7 Claude Sonnet 4.6 GPT-5.5 GPT-5.4-mini Gemini 3.1 Pro Gemini 3 Flash Qwen3-VL-235B-A22B-Instruct

Closest papers by meaning

• EgoBench: An Interactive Egocentric Multimodal Benchmark for Tool-Using Agents (NeurIPS 2026 · similarity 0.89)
• Agent-X: Evaluating Deep Multimodal Reasoning in Vision-Centric Agentic Tasks (ICLR 2026 · similarity 0.88)
• VideoWebArena: Evaluating Long Context Multimodal Agents with Video Understanding Web Tasks (ICLR 2025 · similarity 0.88)
• DeepEyesV2: Toward Agentic Multimodal Model (ICLR 2026 · similarity 0.88)
• AgentVista: Evaluating Multimodal Agent in Ultra-Challenging Realistic Visual Scenarios (NeurIPS 2026 · similarity 0.88)
• AgentStudio: A Toolkit for Building General Virtual Agents (ICLR 2025 · similarity 0.87)