Research map / 3D vision / Face and shape
Pose-Controllable Talking Face Generation by Implicitly Modularized Audio-Visual Representation
Check your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add the audio-visual contrastive synchronization loss (ablation) | PC-AVS · VoxCeleb2 | Mouth LMD: 13.52 -> 6.88 | Helped |
| add the audio-visual contrastive synchronization loss (ablation) | PC-AVS · VoxCeleb2 | SSIM: 0.836 -> 0.886 | Helped |
| use a 12-dimensional pose code instead of a 36-dimensional code (ablation) | PC-AVS · VoxCeleb2 | Mouth LMD: 9.17 -> 6.88 | Helped |
| use a 12-dimensional pose code instead of a 36-dimensional code (ablation) | PC-AVS · VoxCeleb2 | Pose LMD: 9.40 -> 5,9 | Helped |
| use modulated convolution instead of an AdaIN generator (ablation) | PC-AVS · VoxCeleb2 | SSIM: 0.750 -> 0.886 | Helped |
| use modulated convolution instead of an AdaIN generator (ablation) | PC-AVS · VoxCeleb2 | Mouth LMD: 10.58 -> 6.88 | Helped |
| remove target-frame data augmentation (ablation) | PC-AVS · VoxCeleb2 | pose learning | Hurt |
| use traditional skip connections in the generator (ablation) | PC-AVS · VoxCeleb2 | generation quality | Hurt |
| use pose-source control rather than fixing pose | PC-AVS · LRW | LMD: 6.14 -> 3.93 | Helped |
| use pose-source control rather than fixing pose | PC-AVS · VoxCeleb2 | LMD: 7.68 -> 6.88 | Helped |
| use pose-source-controlled PC-AVS rather than Wav2Lip | talking-face generation · VoxCeleb2 user study | Head Movement Naturalness MOS: 1.60 -> 4.20 | Helped |
| use pose-source-controlled PC-AVS rather than MakeitTalk | talking-face generation · VoxCeleb2 user study | Video Realness MOS: 3.22 -> 4.07 | Helped |
Closest papers by meaning
- GoHD: Gaze-oriented and Highly Disentangled Portrait Animation with Rhythmic Poses and Realistic Expressions (AAAI 2025 · similarity 0.89)
- Perceptually Accurate 3D Talking Head Generation: New Definitions, Speech-Mesh Representation, and Evaluation Metrics (CVPR 2025 · similarity 0.88)
- SyncTalk: The Devil is in the Synchronization for Talking Head Synthesis (CVPR 2024 · similarity 0.88)
- Faces that Speak: Jointly Synthesising Talking Face and Speech from Text (CVPR 2024 · similarity 0.88)
- FaceChain-ImagineID: Freely Crafting High-Fidelity Diverse Talking Faces from Disentangled Audio (CVPR 2024 · similarity 0.88)
- ProsodyTalker: 3D Visual Speech Animation via Prosody Decomposition (AAAI 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.