Research map / Vision-language and multimodal / Cross-modal learning
AutoAD II: The Sequel - Who, When, and What in Movie Audio Description
Tengda Han, Max Bain, Arsha Nagrani, Gul Varol, Weidi Xie, Andrew Zisserman
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use cross-attention conditioning instead of prompt-based visual conditioning (ablation) | AD generation LM · MAD-eval-named | CIDEr: 6.7 -> 10.0 | Improved |
| add recognized character names to prompt (ablation) | X-Attn AD model · MAD-eval-named | CIDEr: 10.0 -> 14.2 | Improved |
| add actor names to character prompt (ablation) | X-Attn AD model · MAD-eval-named | CIDEr: 14.2 -> 15.0 | No clear effect |
| add character exemplar image features (ablation) | X-Attn AD model · MAD-eval-named | CIDEr: 15.0 -> 18.3 | Improved |
| use recognized active characters instead of full cast list (ablation) | X-Attn AD model · MAD-eval-named | CIDEr: 14.9 -> 18.3 | Improved |
| add AudioVault and WebVid partial-data pretraining (ablation) | Ours · MAD-eval local | CIDEr: 18.3 -> 19.2 | Improved |
| add previous AD as recurrent context (ablation) | Ours with pretraining · MAD-eval | CIDEr: 19.2 -> 19.5 | Improved |
| use transformer decoder instead of thresholding cosine similarity for character recognition (ablation) | Character recognition module · 4 MAD-eval movies | Average Precision: 0.55 -> 0.87 | Improved |
| train recognition on MovieNet face labels instead of NER-mined labels (ablation) | Character recognition module · 4 MAD-eval movies | Average Precision: 0.74 -> 0.85 | Improved |
| add linear projection layer to character recognition (ablation) | Character recognition module · 4 MAD-eval movies | Average Precision: 0.85 -> 0.87 | Improved |
| add visual features to duration for AD temporal proposal (ablation) | TFM temporal proposal · MAD-t-eval | Average Precision: 0.53 -> 0.61 | Improved |
| add movie subtitles as model input (ablation) | AD generation · MAD-eval | CIDEr | No clear effect |
Closest papers by meaning
- Movie Description (IJCV 2017 · similarity 0.92)
- Contextual AD Narration with Interleaved Multimodal Sequence (CVPR 2025 · similarity 0.92)
- DistinctAD: Distinctive Audio Description Generation in Contexts (CVPR 2025 · similarity 0.91)
- AutoAD III: The Prequel - Back to the Pixels (CVPR 2024 · similarity 0.91)
- MICap: A Unified Model for Identity-Aware Movie Descriptions (CVPR 2024 · similarity 0.85)
- MM-Narrator: Narrating Long-form Videos with Multimodal In-Context Learning (CVPR 2024 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
