AI Paper Atlas

Research map / Video / Video-language understanding

As the Story Unfolds: Watching a Film and Identifying Characters as a Human Does

Zhongrui Gui, Junyu Xie, Tengda Han, Weidi Xie, Andrew Zisserman

NeurIPS 2026 · Video › Video-language understanding · approach: Event and anomaly reasoning in video · 0 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces a character naming task for films, where models must identify principal characters and assign names from the movie itself without external cast lists or photos.

Problem

Identifying principal characters and assigning their names in films without external cast lists, actor photos, or pre-built character banks, since characters are introduced incrementally as the story unfolds.

Approach

Frames character naming as a task: given a name mentioned in dialogue and the associated video context, the model must determine which visible character bears that name, using only information from the movie itself.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
add RL (GRPO) stage after SFT (ablation)Qwen3VL-4B-Thinking · MCN test (character naming)overall accuracy: 64.16 -> 72.76Improved
add color-permutation augmentation during SFT (ablation)Qwen3VL-4B-Thinking SFT · MCN test (character naming)overall accuracy: 62.39 -> 64.16Improved
add color-permutation augmentation during RL (ablation)Qwen3VL-4B-Thinking SFT+RL · MCN test (character naming)overall accuracy: 71.37 -> 72.76Improved
use smaller 4B Thinking with SFT+RL instead of naive SFT on 8B Instruct (ablation)character naming model · MCN test (character naming)overall accuracy: 64.92 -> 72.76Improved
naive fine-tuning of Qwen3VL-8B-Instruct on MCN training setQwen3VL-8B-Instruct · MCN test (character naming)overall accuracy: 53.60 -> 64.92Improved
add face-track filtering before clustering (ablation)streaming character bank · MCN First OccurrencemTIoU_n: 52.27 -> 54.40Improved
add query name filtering (drop off-screen names) (ablation)streaming character bank · MCN First OccurrencePCR@10: 49.46 -> 54.54Improved
add query name filtering (drop off-screen names) (ablation)streaming character bank · MCN First OccurrencemTIoU_v: 85.13 -> 84.90No clear effect
add inter-entry name disambiguation (ablation)streaming character bank · MCN First OccurrencemTIoU_n: 52.40 -> 54.40Improved
add inter-entry name disambiguation (ablation)streaming character bank · MCN First OccurrencemTIoU_v: 84.90 -> 84.90No clear effect
streaming pipeline versus Gemini baselines with full-movie accesscharacter bank construction · MCN First OccurrencePCR@5: 44.00 -> 63.00Improved
use fine-tuned MLLM naming instead of subtitle-only MIL namingcharacter bank construction · MCN First OccurrencemTIoU_n: 33.38 -> 54.40Improved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy