Research map / Video / Video generation
Shot-by-Shot: Film-Grammar-Aware Training-Free Audio Description Generation
Junyu Xie, Tengda Han, Max Bain, Arsha Nagrani, Eshika Khandelwal, Gül Varol, Weidi Xie, Andrew Zisserman
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add shot-based temporal context (2 context shots) with dynamic sampling and shot numbers (ablation) | Training-free AD framework · CMD-AD | CIDEr: 22.4 -> 25.1 | Improved |
| add shot number referral instead of no label or red frame box (ablation) | Training-free AD framework · TV-AD | CIDEr: 27.8 -> 28.9 | Improved |
| use dynamic rather than uniform frame sampling (ablation) | Training-free AD framework · CMD-AD | CIDEr: 24.1 -> 25.1 | Improved |
| inject thread structure into Stage I prompt (ablation) | Training-free AD framework · TV-AD subset | CIDEr: 28.8 -> 30.7 | Improved |
| inject thread structure into Stage I prompt (ablation) | Training-free AD framework · CMD-AD subset | CIDEr: 29.9 -> 30.7 | Improved |
| choose Stage I factors by predicted shot scale rather than all factors (ablation) | Training-free AD framework · CMD-AD | CIDEr: 26.0 -> 26.3 | Improved |
| add facial expression factor to Stage I (AutoAD-Zero style) (ablation) | Training-free AD framework · CMD-AD | CRITIC: 47.4 -> 46.8 | No clear effect |
| use dense spatial DINOv2 features instead of CLS tokens for thread clustering (ablation) | Thread structure prediction · Thread-Safe | AP: 0.786 -> 0.902 | Improved |
| compare temporally nearest frames (Side) rather than all or middle frames (ablation) | Thread structure prediction · Thread-Safe | AP: 0.822 -> 0.902 | Improved |
| swap in GPT-4o for both stages | Training-free AD framework · TV-AD | CIDEr: 31.1 -> 34.2 | Improved |
| generate five candidate ADs jointly and select the best (ablation) | Training-free AD framework · TV-AD | CIDEr: 31.1 -> 51.3 | Improved |
| increase sampling randomness (higher top-p and temperature) for independent candidates (ablation) | Assisted AD generation · CMD-AD | CIDEr: 33.3 -> 37.0 | Improved |
Closest papers by meaning
- AutoAD III: The Prequel - Back to the Pixels (CVPR 2024 · similarity 0.90)
- DistinctAD: Distinctive Audio Description Generation in Contexts (CVPR 2025 · similarity 0.87)
- Contextual AD Narration with Interleaved Multimodal Sequence (CVPR 2025 · similarity 0.87)
- ThinkSound: Chain-of-Thought Reasoning in Multimodal LLMs for Audio Generation and Editing (NeurIPS 2025 · similarity 0.86)
- AVGen-Bench: A Task-Driven Benchmark for Multi-Granular Evaluation of Text-to-Audio-Video Generation (ICML 2026 · similarity 0.86)
- ShotBench: Expert-Level Cinematic Understanding in Vision-Language Models (NeurIPS 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
