AI Paper Atlas

Research map / Video / Video generation

Neural Voice Puppetry: Audio-driven Facial Reenactment

Justus Thies, Mohamed Elgharib, Ayush Tewari, Christian Theobalt, Matthias Nießner

ECCV 2020 · Video › Video generation · approach: Talking face generation · 409 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
use dilated instead of strided convolutions in rendering network (ablation)rendering network · Obama self-reenactment (visual tracking)PSNR: 40.12 -> 41.48Improved
use dilated instead of strided convolutions in rendering network (ablation)rendering network · Obama self-reenactment (audio-based)PSNR: 36.32 -> 36.65Improved
add temporal expression prediction network (ablation)audio-expression estimation · Obama self-reenactmentrelative 2D mouth landmark error: 0.058 -> 0.055Improved
train expression network on multiple targets instead of only Obama (ablation)audio-expression network · multiple target videosprediction coherence: 3145 -> 302750 framesImproved
drive rendering with audio-based expressions instead of visual trackingNeural Voice Puppetry · Obama self-reenactmentPSNR: 41.48 -> 36.65Hurt
drive rendering with audio-based expressions instead of visual trackingNeural Voice Puppetry · Obama self-reenactmentrelative 2D mouth landmark error: 0.022 -> 0.055Hurt
use generalized network instead of person-specific one trained on 14 hoursNeural Voice Puppetry · Obama lip syncSyncNet confidence: 5.9 -> 5.2No clear effect

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy