Research map / Image generation / Text-to-image diffusion
Let the Avatar Talk using Texts without Paired Training Data
Xiuzhe Wu, Yang-Tian Sun, Handi Chen, Hang Zhou, Jingdong Wang, Zhengzhe Liu, Xiaojuan Qi
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add InpaintNet to feature-space 3DMM warping (ablation) | text-driven talking avatar · 256 video pairs | FID: 195.165 -> 187.006 | Improved |
| warp in EG3D feature space rather than image space (InpaintNet) (ablation) | text-driven talking avatar · 256 video pairs | Sync: 2.929 -> 3.907 | Improved |
| feature-space InpaintNet versus image-space InpaintNet (ablation) | text-driven talking avatar · 256 video pairs | FID: 190.006 -> 187.006 | Improved |
| add sync loss Ls to InpaintNet training (ablation) | InpaintNet · 256 video pairs | Sync: 2.863 -> 3.907 | Improved |
| add perceptual loss Lp to reconstruction loss (ablation) | InpaintNet · 256 video pairs | FID: 187.709 -> 187.687 | No clear effect |
| add CLIP consistency loss Lc (ablation) | InpaintNet · 256 video pairs | FIDl: 153.816 -> 151.193 | Improved |
| full loss set versus reconstruction only (ablation) | InpaintNet · 256 video pairs | FIDl: 154.683 -> 149.784 | Improved |
| text-driven EG3D method versus SadTalker | Ours · 256 video pairs | Sync: 5.769 -> 3.907 | Hurt |
| proposed method versus baselines on image quality | Ours · 256 video pairs | FID: 196.733 -> 187.006 | Improved |
Closest papers by meaning
- GAIA: Zero-shot Talking Avatar Generation (ICLR 2024 · similarity 0.90)
- InstructAvatar: Text-Guided Emotion and Motion Control for Avatar Generation (AAAI 2025 · similarity 0.89)
- X-Oscar: A Progressive Framework for High-quality Text-guided 3D Animatable Avatar Generation (ICML 2024 · similarity 0.88)
- AvatarVerse: High-Quality & Stable 3D Avatar Creation from Text and Pose (AAAI 2024 · similarity 0.88)
- Real3D-Portrait: One-shot Realistic 3D Talking Portrait Synthesis (ICLR 2024 · similarity 0.88)
- DreamAvatar: Text-and-Shape Guided 3D Human Avatar Generation via Diffusion Models (CVPR 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
