AI Paper Atlas

Research map / Video / Video generation

ReSyncer: Rewiring Style-based Generator for Unified Audio-Visually Synced Facial Performer

Jiazhi Guan, Zhiliang Xu, Hang Zhou, Kaisiyuan Wang, Shengyi He, Zhanwang Zhang, Borong Liang, Haocheng Feng, Errui Ding, Jingtuo Liu, Jingdong Wang, Youjian Zhao and 1 more

ECCV 2024 · Video › Video generation · approach: Talking face generation · 8 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
add 3D mesh guidance into the generator's feature maps (ablation)ReSyncer Ours-Sync · VoxCeleb2DeltaSync: 2.54 -> 0.88Improved
add 3D mesh guidance into the generator's feature maps (ablation)ReSyncer Ours-Sync · VoxCeleb2LMD: 3.60 -> 3.19Improved
feed mesh content into the W-space vector (ablation)ReSyncer Ours-Sync · VoxCeleb2DeltaSync: 1.02 -> 0.88Improved
increase speaking-style reference length from 50 to 90 frames (ablation)Style-SyncFormer · VoxCeleb2DeltaSync: 0.95 -> 0.88Improved
extend style reference length from 90 to 200 frames (ablation)Style-SyncFormer · VoxCeleb2DeltaSync: 0.88 -> 0.88No clear effect
use ReSyncer instead of StyleSync for generalized lip-syncReSyncer · VoxCeleb2DeltaSync: 1.57 -> 0.88Improved
use ReSyncer face swap versus StyleSwapReSyncer · FaceForensics++ID Retrieval (%): 97.05% -> 96.70%Mixed
use FaceFormer-style one-hot speaker embedding instead of style-injected Style-SyncFormer (ablation)3D dynamics prediction · generalized lip-syncLMD and DeltaSyncImproved
use Hifi3DFace topology instead of BFM (ablation)ReSyncer · lip-syncvisual and sync qualityNo clear effect
personalized fine-tuning versus StyleSync-PReSyncer · Personalized lip-syncDeltaSync: 0.55 vs 0.55No clear effect

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy