Research map / Vision-language and multimodal / Multimodal LLMs
SeriesBench: A Benchmark for Narrative-Driven Drama Series Understanding
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces SeriesBench, a 105-series (1,072-video) benchmark of 28 narrative-understanding tasks for video MLLMs, plus PC-DCoT, a training-free dual-chain (plot event + character temporal) prompting framework that boosts several MLLMs by over 10 points.
Problem
Existing video benchmarks evaluate standalone clips and mostly visual elements, so MLLMs' ability to understand narrative structure, character relationships, and multimodal elements across drama series is untested.
Approach
A long-span narrative annotation method and full-information transformation (GPT-4o generating true/false, multiple-choice and open-ended questions from annotated statements, subtitles, themes and character info), combined with PC-DCoT, which builds a Plot Event Chain and Character Temporal Chain, aligns them by timestamp, and merges them to reason over the video.
Key results
- GPT-4o with PC-DCoT reaches 76.2% overall accuracy on SeriesBench, up 13.4 points from 62.8% without it.
- Qwen2-VL with PC-DCoT improves from 60.3% to 73.9% overall (+13.6).
- Open-source models (e.g., Qwen2-VL 60.3%) trail GPT-4o (62.8%) only modestly, while human performance is 95.8%.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use PC-DCoT at inference | InternVL2 · SeriesBench test | Overall accuracy: 59.2 -> 73.3 | Helped |
| Use PC-DCoT at inference | Qwen2-VL · SeriesBench test | Overall accuracy: 60.3 -> 73.9 | Helped |
| Use PC-DCoT at inference | MiniCPM-V 2.6 · SeriesBench test | Overall accuracy: 59.1 -> 72.0 | Helped |
| Use PC-DCoT at inference | GPT-4o · SeriesBench test | Overall accuracy: 62.8 -> 76.2 | Helped |
| Add the Character Temporal Chain (ablation) | PC-DCoT with InternVL2 · SeriesBench test | Overall accuracy: 68.5 -> 73.3 | Helped |
| Add the Plot Event Chain (ablation) | PC-DCoT with InternVL2 · SeriesBench test | Overall accuracy: 68.8 -> 73.3 | Helped |
| Add the Character Temporal Chain (ablation) | PC-DCoT with Qwen2-VL · SeriesBench test | Overall accuracy: 66.4 -> 73.9 | Helped |
| Add the Plot Event Chain (ablation) | PC-DCoT with Qwen2-VL · SeriesBench test | Overall accuracy: 65.4 -> 73.9 | Helped |
| Add the Character Temporal Chain (ablation) | PC-DCoT with MiniCPM-V 2.6 · SeriesBench test | Overall accuracy: 61.6 -> 72.0 | Helped |
| Add the Plot Event Chain (ablation) | PC-DCoT with MiniCPM-V 2.6 · SeriesBench test | Overall accuracy: 61.3 -> 72.0 | Helped |
| Add the Character Temporal Chain (ablation) | PC-DCoT with GPT-4o · SeriesBench test | Overall accuracy: 69.0 -> 76.2 | Helped |
| Add the Plot Event Chain (ablation) | PC-DCoT with GPT-4o · SeriesBench test | Overall accuracy: 72.3 -> 76.2 | Helped |
Surprising findings
- Qwen2-VL is the only model that improves when earlier episodes are added as context; InternVL2 and MiniCPM-V 2.6 decline.
- Textual subtitles help more than video frames, which the authors interpret as subtitles acting as a screenplay.
Limitations
- Frame sampling disrupts visual continuity, limiting models on narrative tasks.
- Large remaining gap to human performance on causal reasoning and multi-character plot analysis.
- GPT-4o evaluated with only 50 uniformly sampled frames and multi-episode evaluation not feasible for it due to API limits.
Experimental setup
Datasets: SeriesBench
Benchmarks: SeriesBench
Metrics: accuracy BLEU-2 METEOR BERTScore F1
Baselines: InternVL2 LLaVA-OneVision LLaVA-Video Qwen2-VL MiniCPM-V 2.6 Aria VITA VideoLLaMA2.1-AV GPT-4o-mini GPT-4o
Builds on: chain-of-thought Video-MME VideoVista MathVista
Closest papers by meaning
- MovieBench: A Hierarchical Movie Level Dataset for Long Video Generation (CVPR 2025 · similarity 0.87)
- ViStoryBench: Comprehensive Benchmark Suite for Story Visualization (CVPR 2026 · similarity 0.86)
- StoryGPT-V: Large Language Models as Consistent Story Visualizers (CVPR 2025 · similarity 0.86)
- LogiStory: A Logic-Aware Framework for Multi-Image Story Visualization (ICLR 2026 · similarity 0.85)
- VLog: Video-Language Models by Generative Retrieval of Narration Vocabulary (CVPR 2025 · similarity 0.85)
- Previously on ... From Recaps to Story Summarization (CVPR 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.