Research map / Video / Video generation
Zero-1-to-A: Zero-Shot One Image to Animatable Head Avatars Using Video Diffusion
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Zero-1-to-A generates animatable 4D head avatars from a single reference image by iteratively building a spatially and temporally consistent video dataset with a pre-trained video diffusion model and reconstructing a Gaussian head avatar on it, using a simple-to-complex progressive learning schedule.
Problem
Animatable head avatar generation normally needs large human data, and distilling 4D avatars from video diffusion via SDS yields over-smooth results due to spatial and temporal inconsistencies in generated videos.
Approach
SymGEN co-evolves a synthesized video dataset and the avatar: the avatar's renderings guide DDIM-inversion-based refinement of dataset videos, and the refined dataset is then used to reconstruct the avatar. Progressive learning first fixes expression and varies camera (front to side), then fixes camera and varies expression (relaxed to exaggerated).
Key results
- Ours achieves CLIP ViT-L/14 score 0.285 vs 0.275 for HeadStudio (best baseline), and 0.322 ViT-B/32 vs 0.317.
- Rendering speed of 76 FPS vs 48 FPS for HeadStudio on the Geralt example.
- Ablations show removing progressive learning or dataset update yields over-smooth textures and misaligned eyes/mouth.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use Zero-1-to-A instead of HeadStudio | Image-conditioned head avatar generation · CLIP-Score | ViT-L/14 CLIP score: 0.275 -> 0.285 | Helped |
| Use Zero-1-to-A instead of HeadStudio | Image-conditioned head avatar generation · CLIP-Score | ViT-B/16 CLIP score: 0.322 -> 0.320 | Hurt |
| Use Zero-1-to-A instead of HeadStudio | Image-conditioned head avatar generation · CLIP-Score | ViT-B/32 CLIP score: 0.317 -> 0.322 | Helped |
| Add Spatial Consistency Learning to temporal-only learning (ablation) | Zero-1-to-A · Head avatar animation | eye alignment and expression control | Helped |
| Add Temporal Consistency Learning to spatial-only learning (ablation) | Zero-1-to-A · Head avatar animation | generalization to exaggerated expressions | Helped |
| Use Progressive Learning instead of random learning (ablation) | Zero-1-to-A with SymGEN · Head avatar animation | expression control | Helped |
| Use iterative symbiotic dataset updates instead of a one-time dataset update (ablation) | Zero-1-to-A · Head avatar reconstruction | texture detail and animation quality | Helped |
| Use SymGEN reconstruction instead of video-diffusion SDS loss (ablation) | Zero-1-to-A · Head avatar reconstruction | texture detail | Helped |
Surprising findings
- Plain SDS distillation from video diffusion produces over-smooth avatars lacking texture detail.
Limitations
- Cannot model elements beyond the head, such as afro hairstyles, due to FLAME-aligned Gaussians.
- Edge blur may result from underfitting and labeling ambiguities.
Experimental setup
Datasets: ARKit blendshape expressions talk show videos (expression sequences) synthesized video dataset from video diffusion
Benchmarks: CLIP score on static avatar prompts (10 portraits) qualitative dynamic avatar comparison
Metrics: CLIP-Score ViT-L/14 CLIP-Score ViT-B/16 CLIP-Score ViT-B/32 FPS rendering speed
Baselines: DreamFusion LatentNeRF Fantasia3D ProlificDreamer HeadSculpt HeadArtist HeadStudio TADA Follow-Your-Emoji AniPortrait
Builds on: Animatable Gaussian Head score distillation sampling iterative dataset update portrait video diffusion (Hallo-style) ProlificDreamer
Closest papers by meaning
- CAP4D: Creating Animatable 4D Portrait Avatars with Morphable Multi-View Diffusion Models (CVPR 2025 · similarity 0.90)
- Morphable Diffusion: 3D-Consistent Diffusion for Single-image Avatar Creation (CVPR 2024 · similarity 0.90)
- Human-3Diffusion: Realistic Avatar Creation via Explicit 3D Consistent Diffusion Models (NeurIPS 2024 · similarity 0.88)
- UltrAvatar: A Realistic Animatable 3D Avatar Diffusion Model with Authenticity Guided Textures (CVPR 2024 · similarity 0.88)
- DiffusionAvatars: Deferred Diffusion for High-fidelity 3D Head Avatars (CVPR 2024 · similarity 0.88)
- GAF: Gaussian Avatar Reconstruction from Monocular Videos via Multi-view Diffusion (CVPR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.