Research map / 3D vision / NeRF and radiance fields
GeneFace: Generalized and High-Fidelity Audio-Driven 3D Talking Face Synthesis
Zhenhui Ye, Ziyue Jiang, Yi Ren, Jinglin Liu, Jinzheng He, Zhou Zhao
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use GeneFace instead of AD-NeRF | Talking-face synthesis · In-domain audio | FID: 27.52 -> 22.88 | Improved |
| use GeneFace instead of Wav2Lip | Talking-face synthesis · In-domain audio | LMD: 3.988 -> 3.933 | Improved |
| use GeneFace instead of Wav2Lip | Talking-face synthesis · In-domain audio | Sync: 9.212 -> 6.987 | Hurt |
| use GeneFace instead of the strongest listed baseline, Wav2Lip | Talking-face synthesis · OOD audio | FID: 68.05 -> 27.38 | Improved |
| use GeneFace instead of Wav2Lip | Talking-face synthesis · OOD audio | Sync: 9.645 -> 6.212 | Hurt |
| add a flow-based prior rather than using a Gaussian prior (ablation) | GeneFace variational motion generator · In-domain audio | Sync: 6.404 -> 6.987 | Improved |
| add a flow-based prior rather than using a Gaussian prior (ablation) | GeneFace variational motion generator · OOD audio | Sync: 5.831 -> 6.212 | Improved |
| add sync-expert supervision (ablation) | GeneFace variational motion generator · In-domain audio | Sync: 5.972 -> 6.987 | Improved |
| add sync-expert supervision (ablation) | GeneFace variational motion generator · OOD audio | Sync: 5.549 -> 6.212 | Improved |
| add the domain-adaptive post-net (ablation) | GeneFace · In-domain audio | FID: 30.26 -> 22.88 | Improved |
| add the domain-adaptive post-net (ablation) | GeneFace · OOD audio | Sync: 5.248 -> 6.212 | Improved |
| use the domain-adaptive post-net instead of target-person fine-tuning (ablation) | GeneFace · OOD audio | Sync: 5.966 -> 6.212 | Improved |
Closest papers by meaning
- AE-NeRF: Audio Enhanced Neural Radiance Field for Few Shot Talking Head Synthesis (AAAI 2024 · similarity 0.93)
- FaceTalk: Audio-Driven Motion Diffusion for Neural Parametric Head Models (CVPR 2024 · similarity 0.89)
- DAWN: Dynamic Frame Avatar with Non-autoregressive Diffusion Framework for Talking head Video Generation (ICLR 2025 · similarity 0.89)
- GoHD: Gaze-oriented and Highly Disentangled Portrait Animation with Rhythmic Poses and Realistic Expressions (AAAI 2025 · similarity 0.89)
- Real3D-Portrait: One-shot Realistic 3D Talking Portrait Synthesis (ICLR 2024 · similarity 0.89)
- MimicTalk: Mimicking a personalized and expressive 3D talking face in minutes (NeurIPS 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
