Research map / 3D vision / Face and shape
PERSE: Personalized 3D Generative Avatars from A Single Portrait
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Builds a personalized, animatable 3D Gaussian Splatting avatar from a single portrait with disentangled, continuous facial-attribute latent control, trained on a synthetic attribute-edited video dataset and regularized with DiffMorpher-interpolated 2D faces.
Problem
Creating personalized 3D generative avatars from a single reference portrait that allow editing of individual facial attributes (hair, beard, hat, etc.) in a continuous, disentangled latent space while preserving identity, which prior methods cannot do without multi-subject video data.
Approach
Synthesizes attribute-edited 2D monocular videos of a single reference identity (inpainting with FLUX/SDXL plus animation with LivePortrait and a newly trained portrait-Champ image-to-video model), trains a PEGASUS-style part-latent avatar in 3D Gaussian Splatting with CLIP-conditioned latents, and regularizes interpolated latents using DiffMorpher-generated 2D pseudo supervision.
Key results
- On unseen pose renderings, PERSE reaches PSNR 23.84, SSIM 0.8852, LPIPS 0.1458 and identity 0.7059, outperforming PEGASUS (23.56 PSNR, identity 0.6471).
- On interpolated renderings, PERSE achieves FID_FFHQ 214.46 vs. 223.86 for PEGASUS and FID_syn 57.78 vs. 84.94, with a user-study preference of 63.7 vs. 36.3.
- Avatar construction takes approximately 1.5 days on eight RTX A6000 GPUs per identity.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use PERSE instead of PEGASUS | Personalized avatar model · Unseen pose renderings | PSNR: 23.56 -> 23.84 | Helped |
| Use PERSE instead of Cond.SA | Personalized avatar model · Unseen pose renderings | SSIM: 0.8690 -> 0.8852 | Helped |
| Use PERSE instead of PEGASUS | Personalized avatar model · Unseen pose renderings | Identity: 0.6471 -> 0.7059 | Helped |
| Use PERSE instead of PEGASUS | Personalized avatar model · Interpolated renderings | FIDFFHQ: 223.86 -> 214.46 | Helped |
| Use PERSE instead of PEGASUS | Personalized avatar model · Interpolated renderings | PPL: 0.4047 -> 0.3308 | Helped |
| Add interpolation loss supervision (ablation) | PERSE · Interpolated renderings | FIDsyn: 66.34 -> 57.78 | Helped |
| Add interpolation loss supervision (ablation) | PERSE · Interpolated renderings | Identity: 0.7001 -> 0.7013 | Helped |
| Add CLIP-guided latent conditioning (ablation) | PERSE · Interpolated renderings | FIDsyn: 69.03 -> 57.78 | Helped |
| Add CLIP-guided latent conditioning (ablation) | PERSE · Interpolated renderings | PPL: 0.3268 -> 0.3308 | Mixed |
Surprising findings
- PEGASUS achieves slightly better PDV than PERSE, despite worse FID, KID and PPL on interpolation.
Limitations
- Avatar building is computationally intensive, about 1.5 days on eight GPUs per identity.
- Avatars do not yet reach photorealism, especially fine hair strand details.
- Unseen attributes are not handled perfectly without per-attribute LoRA fine-tuning.
Experimental setup
Datasets: synthetic attribute-edited portrait videos 6k real-world videos for portrait-Champ FFHQ
Benchmarks: PERSE synthetic attribute-edited video dataset FFHQ
Metrics: PSNR SSIM LPIPS ArcFace identity similarity FID KID PDV PPL user study
Baselines: PEGASUS Conditional INSTA Conditional SplattingAvatar
Builds on: PEGASUS 3D Gaussian Splatting Champ DiffMorpher LivePortrait
Closest papers by meaning
- PEGASUS: Personalized Generative 3D Avatars with Composable Attributes (CVPR 2024 · similarity 0.92)
- Creating Your Editable 3D Photorealistic Avatar with Tetrahedron-constrained Gaussian Splatting (CVPR 2025 · similarity 0.88)
- Arc2Avatar: Generating Expressive 3D Avatars from a Single Image via ID Guidance (CVPR 2025 · similarity 0.88)
- AttriHuman-3D: Editable 3D Human Avatar Generation with Attribute Decomposition and Indexing (CVPR 2024 · similarity 0.88)
- En3D: An Enhanced Generative Model for Sculpting 3D Humans from 2D Synthetic Data (CVPR 2024 · similarity 0.87)
- Low-Rank Head Avatar Personalization with Registers (NeurIPS 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.