Research map / Video / Video generation
EMO: Emote Portrait Alive - Generating Expressive Portrait Videos with Audio2Video Diffusion Model under Weak Conditions
Linrui Tian, Qi Wang, Bang Zhang, Liefeng Bo
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add speed layers with an assigned speed value (ablation) | EMO · HDTF | Velocity Variance: 1.002 -> 0.357 | Improved |
| add speed layers (ablation) | EMO · HDTF | VMV: 0.134 -> 0.046 | Improved |
| raise assigned speed value from 0.1 to 1.9 (ablation) | EMO speed layers · HDTF | Mean Velocity: 0.878 -> 1.246 | Improved |
| add 250 hours of collected training data (ablation) | EMO · HDTF | E-FID: 0.215 -> 0.116 | Improved |
| add 250 hours of collected training data (ablation) | EMO · HDTF | FVD: 102.78 -> 67.66 | Improved |
| use direct audio-to-video diffusion instead of 3DMM or landmark intermediates | talking head generation · HDTF | E-FID: 0.503 -> 0.116 (vs SadTalker) | Improved |
| EMO versus Wav2Lip on lip sync | talking head generation · HDTF | SyncNet: 5.79 -> 3.89 | Hurt |
| train speed layers jointly with audio layers (ablation) | EMO · training strategy | audio driving ability | Hurt |
Closest papers by meaning
- EmotiveTalk: Expressive Talking Head Generation through Audio Information Decoupling and Emotional Video Diffusion (CVPR 2025 · similarity 0.90)
- VASA-1: Lifelike Audio-Driven Talking Faces Generated in Real Time (NeurIPS 2024 · similarity 0.88)
- GoHD: Gaze-oriented and Highly Disentangled Portrait Animation with Rhythmic Poses and Realistic Expressions (AAAI 2025 · similarity 0.88)
- Style2Talker: High-Resolution Talking Head Generation with Emotion Style and Art Style (AAAI 2024 · similarity 0.88)
- Hallo3: Highly Dynamic and Realistic Portrait Image Animation with Video Diffusion Transformer (CVPR 2025 · similarity 0.88)
- KeyFace: Expressive Audio-Driven Facial Animation for Long Sequences via KeyFrame Interpolation (CVPR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
