AI Paper Atlas

Research map / Image generation / Text-to-image diffusion

Let the Avatar Talk using Texts without Paired Training Data

Xiuzhe Wu, Yang-Tian Sun, Handi Chen, Hang Zhou, Jingdong Wang, Zhengzhe Liu, Xiaojuan Qi

ECCV 2024 · Image generation › Text-to-image diffusion · approach: Layout-to-image control

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
add InpaintNet to feature-space 3DMM warping (ablation)text-driven talking avatar · 256 video pairsFID: 195.165 -> 187.006Improved
warp in EG3D feature space rather than image space (InpaintNet) (ablation)text-driven talking avatar · 256 video pairsSync: 2.929 -> 3.907Improved
feature-space InpaintNet versus image-space InpaintNet (ablation)text-driven talking avatar · 256 video pairsFID: 190.006 -> 187.006Improved
add sync loss Ls to InpaintNet training (ablation)InpaintNet · 256 video pairsSync: 2.863 -> 3.907Improved
add perceptual loss Lp to reconstruction loss (ablation)InpaintNet · 256 video pairsFID: 187.709 -> 187.687No clear effect
add CLIP consistency loss Lc (ablation)InpaintNet · 256 video pairsFIDl: 153.816 -> 151.193Improved
full loss set versus reconstruction only (ablation)InpaintNet · 256 video pairsFIDl: 154.683 -> 149.784Improved
text-driven EG3D method versus SadTalkerOurs · 256 video pairsSync: 5.769 -> 3.907Hurt
proposed method versus baselines on image qualityOurs · 256 video pairsFID: 196.733 -> 187.006Improved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy