Research map / Video / Video generation
One-Shot Talking Face Generation from Single-Speaker Audio-Visual Correlation Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| train the audio-visual correlation model on one speaker and transfer its learned speaking style | one-shot talking face generation · HDTF | LMD: 0.289 -> 0.271 | Helped |
| train the audio-visual correlation model on one speaker and transfer its learned speaking style | one-shot talking face generation · VoxCeleb2 | AVOff: 0.50 -> -0.08 | Helped |
| add phoneme input to the encoder (ablation) | AVCT · HDTF | LMD: 0.376 -> 0.274 | Helped |
| add audio features to the decoder (ablation) | AVCT · HDTF | AVConf: 6.59 -> 7.17 | Helped |
| add keypoint features to the decoder (ablation) | AVCT · HDTF | LMD: 0.418 -> 0.274 | Helped |
| use extracted keypoint features instead of the reference image (ablation) | AVCT · HDTF | FID: 0.210 -> 0.148 | Helped |
| add temporal and lip-sync discriminators through batched sequential training (ablation) | AVCT · HDTF | AVConf: 4.55 -> 7.17 | Helped |
| add the lip-sync discriminator (ablation) | AVCT · HDTF | AVConf: 5.53 -> 7.17 | Helped |
Closest papers by meaning
- Faces that Speak: Jointly Synthesising Talking Face and Speech from Text (CVPR 2024 · similarity 0.90)
- Synergizing Motion and Appearance: Multi-Scale Compensatory Codebooks for Talking Head Video Generation (CVPR 2025 · similarity 0.89)
- OmniTalker: One-shot Real-time Text-Driven Talking Audio-Video Generation With Multimodal Style Mimicking (NeurIPS 2025 · similarity 0.89)
- VASA-1: Lifelike Audio-Driven Talking Faces Generated in Real Time (NeurIPS 2024 · similarity 0.89)
- From Faces to Voices: Learning Hierarchical Representations for High-quality Video-to-Speech (CVPR 2025 · similarity 0.89)
- GoHD: Gaze-oriented and Highly Disentangled Portrait Animation with Rhythmic Poses and Realistic Expressions (AAAI 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.