AI Paper Atlas

Research map / Video / Video generation

Speech2Lip: High-fidelity Speech to Lip Generation by Learning from a Short Video

Xiuzhe Wu, Pengfei Hu, Yang Wu, Xiaoyang Lyu, Yan-Pei Cao, Ying Shan, Wenming Yang, Zhongqian Sun, Xiaojuan Qi

ICCV 2023 · Video › Video generation · approach: Talking face generation · 13 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
add depth loss L_d to whole-image reconstruction loss (ablation)Speech2Lip · Testset ISync: 6.313 -> 6.359No clear effect
add canonical mouth reconstruction loss L_m (ablation)Speech2Lip · Testset ISync: 6.359 -> 7.194Improved
add contrastive sync loss L_s (ablation)Speech2Lip · Testset ISync: 7.194 -> 7.771Improved
include unsynced negative speech-image pairs in the contrastive sync loss (ablation)Speech2Lip · Testset III (cross-identity)Sync: 3.830 -> 4.379Improved
add Blend-Net (ablation)Speech2Lip · Testset IPSNR (lip): 33.736 -> 34.815Improved
add Blend-Net (ablation)Speech2Lip · Testset ISync: 6.065 -> 7.771Improved
use implicit model instead of explicit lip generator (ablation)Speech2Lip · Testset ISync: 6.790 -> 7.771Improved
add timestamp input (ablation)Speech2Lip · Testset ISync: 7.146 -> 7.771Improved
add continuous sampling (ablation)Speech2Lip · Testset ISync: 7.553 -> 7.771Improved
use Speech2Lip instead of speaker-independent Wav2Lip on Synctalking head synthesis · Testset ISync: 10.031 -> 7.771Hurt

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy