Research map / 3D vision / Face and shape
Synthetic Prior for Few-Shot Drivable Head Avatar Inversion
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
SynShot builds a drivable 3D Gaussian head avatar prior purely from a 14M-image synthetic dataset via a VQ-VAE in UV space, then few-shot inverts real subjects from about three images using pivotal tuning.
Problem
Few-shot creation of photorealistic, drivable head avatars that generalize to novel expressions and viewpoints, without relying on costly or privacy-restricted real multi-view data.
Approach
Train a convolutional VQ-VAE-based generative 3D Gaussian head prior on a large synthetic head dataset with disentangled identity and expression latents, predicting per-part Gaussian maps in UV space, then fine-tune decoders and regressors via pivotal tuning on a few real images.
Key results
- SynShot with 3 input views outperforms monocular methods INSTA, Flash Avatar, and Splatting Avatar trained on ~3000 frames on cross-reenactment.
- Self-reenactment LPIPS: SynShot 0.0236 vs InvertAvatar 0.0962, Portrait4D 0.0843, Next3D 0.2274.
- Inversion with up to 20 views takes under 10 minutes on a single H100.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use SynShot instead of GAN-based inversion baselines | GAN-based head avatar self-reenactment · GAN-based self-reenactment | LPIPS: 0.0236 vs 0.0962 (InvertAvatar), 0.0843 (Portrait4D), and 0.2274 (Next3D) | Helped |
| predict feature maps at 256×256 and bilinearly upscale sampled maps to 512 × 512 (ablation) | SynShot VQ-VAE architecture · 50 held-out test actors | LPIPS: 0.2686 -> 0.2695 | Hurt |
| use a single layer instead of separate hair and face layers (ablation) | SynShot VQ-VAE architecture · 50 held-out test actors | LPIPS: 0.2686 -> 0.2702 | Hurt |
| reduce feature-map channels to F = 32 (ablation) | SynShot VQ-VAE architecture · 50 held-out test actors | LPIPS: 0.2686 -> 0.2732 | Hurt |
| remove codebook quantization (ablation) | SynShot VQ-VAE architecture · 50 held-out test actors | LPIPS: 0.2686 -> 0.2747 | Hurt |
| reduce feature-map channels to F = 64 (ablation) | SynShot VQ-VAE architecture · 50 held-out test actors | LPIPS: 0.2686 -> 0.2765 | Hurt |
| regress Gaussian primitives directly without texture sampling (ablation) | SynShot VQ-VAE architecture · 50 held-out test actors | LPIPS: 0.2686 -> 0.2853 | Hurt |
| use 256 × 256 final textures instead of 512 × 512 (ablation) | SynShot VQ-VAE architecture · 50 held-out test actors | LPIPS: 0.2686 -> 0.2865 | Hurt |
| remove LId and LArcFeat (ablation) | SynShot inversion pipeline · Inversion reconstruction | LPIPS: 0.0776 -> 0.0839 | Hurt |
| remove LArcFeat (ablation) | SynShot inversion pipeline · Inversion reconstruction | LPIPS: 0.0776 -> 0.0848 | Hurt |
| use only L1 and SSIM photometric losses (ablation) | SynShot inversion pipeline · Inversion reconstruction | LPIPS: 0.0776 -> 0.0904 | Hurt |
Surprising findings
- Monocular methods trained on thousands of frames can be matched or beaten by a synthetic-prior inversion with only three images.
- Evaluating on limited-motion sequences hides generalization failures; about 13 frames may suffice for monocular methods on the standard test set.
Limitations
- Domain gap between synthetic and real data; teeth geometry and texture are shared across synthetic subjects.
- Synthetic data lacks expression-dependent wrinkles and uses a single environment map for lighting.
- Relies on a 3DMM regressor and input images to extract a texture for initialization at test time.
Experimental setup
Datasets: synthetic head dataset (2,000 identities, 14M images) monocular sequences from INSTA, and three other datasets
Benchmarks: monocular self-reenactment (11 sequences) cross-reenactment GAN-based self- and cross-reenactment comparisons
Metrics: LPIPS SSIM L1 PSNR
Baselines: INSTA Flash Avatar Splatting Avatar PanoHead HeadNeRF MofaNeRF Portrait4D Next3D InvertAvatar
Builds on: 3D Gaussian splatting VQ-VAE VQGAN pivotal tuning INSTA
Closest papers by meaning
- Vid2Avatar-Pro: Authentic Avatar from Videos in the Wild via Universal Prior (CVPR 2025 · similarity 0.89)
- GASP: Gaussian Avatars with Synthetic Priors (CVPR 2025 · similarity 0.89)
- GAF: Gaussian Avatar Reconstruction from Monocular Videos via Multi-view Diffusion (CVPR 2025 · similarity 0.89)
- High-Fidelity 3D Head Avatars Reconstruction through Spatially-Varying Expression Conditioned Neural Radiance Field (AAAI 2024 · similarity 0.88)
- RGBAvatar: Reduced Gaussian Blendshapes for Online Modeling of Head Avatars (CVPR 2025 · similarity 0.88)
- GeneAvatar: Generic Expression-Aware Volumetric Head Avatar Editing from a Single Image (CVPR 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.