Research map / Video / Video-language understanding
Towards Authentic Movie Dubbing with Retrieve-Augmented Director-Actor Interaction Learning
Rui Liu, Yuan Zhao, Zhenqi Jia
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use LLM-generated scene and face emotion captions instead of I3D/EmoFan embeddings (ablation) | Authentic-Dubber · V2C-Animation test | EMO-ACC: 46.02 -> 47.21 | Improved |
| add emotion-similarity retrieval augmentation (all modalities) (ablation) | Authentic-Dubber · V2C-Animation test | EMO-ACC: 45.23 -> 47.21 | Improved |
| add text retrieval (ablation) | Authentic-Dubber · V2C-Animation test | MOS-DE: 3.540 -> 3.792 | Improved |
| add direct emotion audio nodes in progressive graph (ablation) | Authentic-Dubber · V2C-Animation test | EMO-ACC: 45.30 -> 47.21 | Improved |
| add indirect multimodal information (ablation) | Authentic-Dubber · V2C-Animation test | MOS-DE: 3.542 -> 3.792 | Improved |
| add graph-based modeling (ablation) | Authentic-Dubber · V2C-Animation test | EMO-ACC: 45.92 -> 47.21 | Improved |
| add construct-and-encode paradigm (ablation) | Authentic-Dubber · V2C-Animation test | EMO-ACC: 46.85 -> 47.21 | Improved |
| add hierarchical aggregation (ablation) | Authentic-Dubber · V2C-Animation test | EMO-ACC: 46.71 -> 47.21 | Improved |
| use speaker-agnostic instead of speaker-specific retrieval (ablation) | Authentic-Dubber · V2C-Animation test | EMO-ACC: peak 47.21 at K=3 | Improved |
| increase retrieved Top-K beyond optimum (ablation) | Authentic-Dubber · V2C-Animation test | EMO-ACC | Hurt |
| increase retrieval footage scale from 10% to 100% (ablation) | Authentic-Dubber · V2C-Animation test | EMO-ACC: peak 47.21 | Improved |
| use cosine similarity instead of dot product or Euclidean distance for retrieval (ablation) | Authentic-Dubber · V2C-Animation test | EMO-ACC | Improved |
Closest papers by meaning
- InstructDubber: Instruction-based Alignment for Zero-shot Movie Dubbing (AAAI 2026 · similarity 0.89)
- EmoDubber: Towards High Quality and Emotion Controllable Movie Dubbing (CVPR 2025 · similarity 0.88)
- Prosody-Enhanced Acoustic Pre-training and Acoustic-Disentangled Prosody Adapting for Movie Dubbing (CVPR 2025 · similarity 0.86)
- Style2Talker: High-Resolution Talking Head Generation with Emotion Style and Art Style (AAAI 2024 · similarity 0.84)
- AVERE: Improving Audiovisual Emotion Reasoning with Preference Optimization (ICLR 2026 · similarity 0.83)
- PAI-Actor: Cinematic Multi-Actor Character Replacement in Dynamic Scenes (NeurIPS 2026 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
