Research map / 3D vision / Face and shape
PEGASUS: Personalized Generative 3D Avatars with Composable Attributes
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Builds a personalized generative 3D face avatar from monocular video by synthesizing part-swapped training videos of the target identity and training a latent-code-conditioned PointAvatar-based model, plus a zero-shot part-transfer variant.
Problem
Existing monocular 3D face avatar methods reconstruct an exact replica of the input person and cannot selectively alter facial attributes (hair, nose, hats) while preserving identity; generative 2D/3D face models are not animatable or identity-preserving.
Approach
Synthesize a target-identity video collection with varied facial parts by swapping parts rendered from animatable avatars of other monocular videos, then train a single point-based avatar with disentangled per-part latent codes and a generic canonical space; a zero-shot variant composes point clouds from two avatars without extra training.
Key results
- Oursswap+PA achieves best FID of 169.54 and ArcFace 0.7179 among part-swapping baselines (CD+PA FID 181.60, E4S+PA 176.64, DELTA 198.40).
- Ourszero-shot attains the highest ArcFace identity score of 0.7792 in the hair transfer comparison.
- Multi-stage canonical space improves PSNR from 20.92 to 21.75 over single-stage PointAvatar-style baseline in ablation.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add synthetic part-swapped videos and interpolate latent codes (ablation) | PEGASUS generative avatar · Latent-code appearance generation | FID: 240.17 -> 206.87 | Helped |
| add synthetic part-swapped videos and interpolate latent codes (ablation) | PEGASUS generative avatar · Latent-code appearance generation | KID: 0.2482 -> 0.1839 | Helped |
| add synthetic part-swapped videos and interpolate latent codes (ablation) | PEGASUS generative avatar · Latent-code appearance generation | ArcFace: 0.4722 -> 0.8127 | Helped |
| add generic-to-subject and subject-to-FLAME-canonical offsets (ablation) | PEGASUS · Unseen poses and expressions | PSNR: 20.92 -> 21.75 | Helped |
| add generic-to-subject offset (ablation) | PEGASUS with subject-specific offset · Unseen poses and expressions | SSIM: 0.9033 -> 0.9059 | No clear effect |
| add generic-to-subject offset (ablation) | PEGASUS with subject-specific offset · Unseen poses and expressions | LPIPS: 0.1292 -> 0.1291 | Helped |
| use PEGASUS part-swapped videos followed by PointAvatar instead of DELTA | Hair transfer avatar · Hair transfer | FID: 198.40 -> 169.54 | Helped |
| use PEGASUS zero-shot transfer instead of PEGASUS person-gen | PEGASUS · Hair transfer | ArcFace: 0.6883 -> 0.7792 | Helped |
| use PEGASUS person-gen instead of E4S + PointAvatar | Hair transfer avatar · Hair transfer | ArcFace: 0.5701 -> 0.6883 | Helped |
Surprising findings
- Custom Diffusion + PointAvatar gets a better KID score but changes identity significantly, showing low ArcFace.
Limitations
- Avatar quality does not reach photo-realistic level, with noticeable artifacts.
- Non-physically-based synthetic DB generation limits physical accuracy.
- Single model represents limited identity diversity; extremely diverse individuals yield blurry avatars.
Experimental setup
Datasets: NeRFACE videos 109 internet monocular videos (attribute DB)
Benchmarks: NeRFACE target videos
Metrics: FID KID ArcFace similarity PSNR SSIM LPIPS
Baselines: DELTA E4S + PointAvatar Custom Diffusion + PointAvatar PointAvatar
Builds on: PointAvatar FLAME DECA Poisson Blending
Closest papers by meaning
- PERSE: Personalized 3D Generative Avatars from A Single Portrait (CVPR 2025 · similarity 0.92)
- En3D: An Enhanced Generative Model for Sculpting 3D Humans from 2D Synthetic Data (CVPR 2024 · similarity 0.88)
- Arc2Avatar: Generating Expressive 3D Avatars from a Single Image via ID Guidance (CVPR 2025 · similarity 0.88)
- Low-Rank Head Avatar Personalization with Registers (NeurIPS 2025 · similarity 0.87)
- Morphable Diffusion: 3D-Consistent Diffusion for Single-image Avatar Creation (CVPR 2024 · similarity 0.86)
- Synthetic Prior for Few-Shot Drivable Head Avatar Inversion (CVPR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.