Research map / Video / Video generation
SyncTalkFace: Talking Face Generation with Precise Lip-Syncing via Audio-Lip Memory
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add visual-visual sync loss to audio-visual sync loss (ablation) | SyncTalkFace · LRW and LRS2 test | PSNR, SSIM, LMD, LSE-D, and LSE-C: LRW: PSNR 33.099 -> 33.126, SSIM 0.886 -> 0.893, LMD 1.276 -> 1.253, LSE-D 7.375 -> 7.013, LSE-C 6.162 -> 6.619; LRS2: P | Mixed |
| Add audio-visual sync loss to visual-visual sync loss (ablation) | SyncTalkFace · LRW and LRS2 test | PSNR, SSIM, LMD, LSE-D, and LSE-C: LRW: PSNR 33.112 -> 33.126, SSIM 0.893 -> 0.893, LMD 1.262 -> 1.253, LSE-D 7.394 -> 7.013, LSE-C 6.131 -> 6.619; LRS2: P | Mixed |
| Use visual-visual sync loss instead of audio-visual sync loss (ablation) | SyncTalkFace · LRW and LRS2 test | LSE-D and LSE-C: LRW LSE-D 7.375 -> 7.394, LSE-C 6.162 -> 6.131; LRS2 LSE-D 6.392 -> 6.787, LSE-C 7.835 -> 7.363 | Hurt |
| Use visual-visual sync loss instead of audio-visual sync loss (ablation) | SyncTalkFace · LRW and LRS2 test | PSNR, SSIM, and LMD: LRW PSNR 33.099 -> 33.112, SSIM 0.886 -> 0.893, LMD 1.276 -> 1.262; LRS2 PSNR 32.681 -> 32.611, SSIM 0.875 -> 0.875, LMD | Helped |
| Set memory slot count to 24 instead of 96 (ablation) | SyncTalkFace · LRS2 test | PSNR: 32.529 -> 32.522 | Hurt |
| Set memory slot count to 48 instead of 96 (ablation) | SyncTalkFace · LRS2 test | PSNR: 32.529 -> 32.373 | Hurt |
| Increase memory slot count from 96 to 120 (ablation) | SyncTalkFace · LRS2 test | PSNR: 32.529 -> 32.655 | Helped |
| Increase memory slot count from 96 to 120 (ablation) | SyncTalkFace · LRS2 test | SSIM, LMD, LSE-D, and LSE-C: SSIM 0.876 -> 0.873; LMD 1.387 -> 1.469; LSE-D 6.352 -> 6.475; LSE-C 7.925 -> 7.785 | Hurt |
| Use Audio-Lip Memory to provide lip-motion visual hints to the decoder | SyncTalkFace · LRW and LRS2 test | PSNR, SSIM, and LMD | Helped |
Closest papers by meaning
- Perceptually Accurate 3D Talking Head Generation: New Definitions, Speech-Mesh Representation, and Evaluation Metrics (CVPR 2025 · similarity 0.92)
- PointTalk: Audio-Driven Dynamic Lip Point Cloud for 3D Gaussian-based Talking Head Synthesis (AAAI 2025 · similarity 0.90)
- OmniSync: Towards Universal Lip Synchronization via Diffusion Transformers (NeurIPS 2025 · similarity 0.90)
- ProsodyTalker: 3D Visual Speech Animation via Prosody Decomposition (AAAI 2025 · similarity 0.89)
- Long-Term TalkingFace Generation via Motion-Prior Conditional Diffusion Model (ICML 2025 · similarity 0.89)
- Seeing Speech: Learning Visible Articulatory Dynamics for Speech-Driven 3D Facial Animation (NeurIPS 2026 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.