Research map / 3D vision / Face and shape
3DToonify: Creating Your High-Fidelity 3D Stylized Avatar Easily from 2D Portrait Images
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes 3DToonify, a progressive-training NeRF/SDF-based spatial neural representation that turns a monocular portrait video plus one stylized exemplar into a view-consistent 3D stylized avatar.
Problem
Generating high-fidelity, view-consistent 3D stylized avatars from a monocular portrait video guided by a single style exemplar, which naive 2D stylization plus 3D reconstruction handles poorly.
Approach
Learn a photorealistic neural implicit prior from the video with MVS depth guidance, then adapt disentangled SDF geometry and a view-independent albedo texture field to the style domain in three progressive stages, using 2D stylized frames from DCT-Net and VToonify as supervision.
Key results
- Achieves FID 57.6 vs 66.5 (DeformToon3D) and 78.8 (NeRF-Art) on stylized renderings, with IP 0.678 vs 0.551 and 0.671.
- Versus 2D methods, FID 57.6 vs 86.9 (VToonify) and 126.1 (DCT-Net), with 3D validity 1.00 vs 0.62 and 0.54.
- Full model FID 94.7 in ablation, better than removing prior (98.7), geometry adaption (105.2), or texture adaption (96.7).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use 3DToonify instead of DeformToon3D | 3D avatar stylization · 3D avatar stylization comparison | FID: 66.5 -> 57.6 | Helped |
| Use 3DToonify instead of NeRF-Art | 3D avatar stylization · 3D avatar stylization comparison | IP: 0.671 -> 0.678 | Helped |
| Train 3DToonify with DCT-Net stylizations instead of using DCT-Net alone | DCT-Net stylization · 2D stylization comparison (DCT-Net) | FID: 126.1 -> 94.7 | Helped |
| Train 3DToonify with VToonify stylizations instead of using VToonify alone | VToonify stylization · 2D stylization comparison (VToonify) | 3D validity: 0.62 -> 1.00 | Helped |
| Add photorealistic prior learning (ablation) | 3DToonify · Progressive training ablation | FID: 98.7 -> 94.7 | Helped |
| Add geometry adaption (ablation) | 3DToonify · Progressive training ablation | FID: 105.2 -> 94.7 | Helped |
| Add texture adaption (ablation) | 3DToonify · Progressive training ablation | FID: 96.7 -> 94.7 | Helped |
| Add progressive structure adaption (ablation) | 3DToonify · Progressive training ablation | FID: 96.2 -> 94.7 | Helped |
| Add MVS guidance during prior learning (ablation) | 3DToonify · Real-world surface reconstruction | surface robustness | Helped |
| Add style enhancement to the extracted texture map (ablation) | 3DToonify · Explicit texture map stylization | stylization vividness | Helped |
Limitations
- Requires a static subject recorded as a video with the person standing still.
- Depends on MVS depth priors (OpenMVS) and 2D stylizers to supply supervision frames.
- Training takes about 20 hours per avatar on one V100.
Experimental setup
Datasets: Portrait360
Benchmarks: Portrait360
Metrics: FID identity preservation (ArcFace similarity) 3D validity
Baselines: DeformToon3D NeRF-Art VToonify DCT-Net
Builds on: NeRF NeuS VToonify DCT-Net OpenMVS
Closest papers by meaning
- LeGO: Leveraging a Surface Deformation Network for Animatable Stylized Face Generation with One Example (CVPR 2024 · similarity 0.90)
- Styl3R: Instant 3D Stylized Reconstruction for Arbitrary Scenes and Styles (NeurIPS 2025 · similarity 0.89)
- CAP4D: Creating Animatable 4D Portrait Avatars with Morphable Multi-View Diffusion Models (CVPR 2025 · similarity 0.88)
- Human-3Diffusion: Realistic Avatar Creation via Explicit 3D Consistent Diffusion Models (NeurIPS 2024 · similarity 0.88)
- DiffusionGAN3D: Boosting Text-guided 3D Generation and Domain Adaptation by Combining 3D GANs and Diffusion Priors (CVPR 2024 · similarity 0.88)
- High-Fidelity 3D Head Avatars Reconstruction through Spatially-Varying Expression Conditioned Neural Radiance Field (AAAI 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.