Research map / 3D vision / Face and shape
WildAvatar: Learning In-the-wild 3D Avatars from the Web
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Builds an automatic four-stage annotation pipeline with filtering protocols over YouTube videos and releases WildAvatar, a 10k+ subject in-the-wild 3D avatar dataset, benchmarked with per-subject and generalizable avatar methods.
Problem
Avatar creation methods depend on costly lab-captured multi-view data that lacks real-world diversity, while web human videos lack reliable SMPL, camera, and mask annotations and vary in quality.
Approach
Combine off-the-shelf detectors, SAM segmentation, and single-image SMPL estimators with ensemble agreement and cross-consistency filters (2D keypoint PCK, SMPL-vs-SAM mask overlap) plus a temporal refinement stage to automatically curate high-confidence in-the-wild human clips at scale.
Key results
- The four-stage annotation pipeline reaches MPJPE 94.9 and PVE 115.5 on EMDB, beating HMR2.0 (98.0 / 120.3).
- Filtering raises PCK from 0.282 to 0.921 and lowers SSIOU from 0.760 to 0.028 on web clips, yielding 10,647 retained clips.
- Training generalizable avatar methods on WildAvatar improves WildAvatar-test PSNR/SSIM/LPIPS by 4.52%, 3.53%, and 5.06% on average.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add alignment and temporal smoothing in Stage IV | WildAvatar annotation pipeline · EMDB | MPJPE: 109.9 -> 94.9 | Helped |
| use WildAvatar annotations instead of HMR2.0 annotations | per-subject avatar creation methods on WildAvatar · WildAvatar | PSNR: +3.51% | Helped |
| add WildAvatar to the training set (ablation) | NHP · WildAvatar test, novel pose | PSNR(NP): 17.39 -> 18.12 | Helped |
| add WildAvatar to the training set (ablation) | MPS-NeRF · WildAvatar test, novel pose | PSNR(NP): 18.21 -> 18.79 | Helped |
| add WildAvatar to the training set (ablation) | SHERF · WildAvatar test, novel pose | PSNR(NP): 18.39 -> 19.43 | Helped |
| add WildAvatar to the training set (ablation) | Generalizable GS · WildAvatar test, novel pose | PSNR(NP): 17.76 -> 18.43 | Helped |
| apply Protocol I filtering for clear bodies and significant movement (ablation) | WildAvatar filtering pipeline · web videos | PCK: 0.282 -> 0.762 | Helped |
| add Protocol II ensemble of annotating experts (ablation) | WildAvatar filtering pipeline · web videos | PCK: 0.762 -> 0.839 | Helped |
| add Protocol III keypoint consistency filtering (ablation) | WildAvatar filtering pipeline · web videos | PCK: 0.839 -> 0.882 | Helped |
| add Protocol IV SMPL and SAM mask consistency filtering (ablation) | WildAvatar filtering pipeline · web videos | SOIOU: 0.129 -> 0.052 | Helped |
| refine SMPL annotations with Stage IV (ablation) | WildAvatar filtering pipeline · web videos | PCK: 0.902 -> 0.921 | Helped |
Surprising findings
- Generalizable 3D Gaussian Splatting performs slightly worse than NeRF-based methods in the generalizable setting, which the authors attribute to difficulty of point cloud initialization from a single image.
Limitations
- Per-subject experiments use only 133 manually selected subjects due to compute cost.
- Annotations lack ground truth; quality is verified only via consistency checks, and PCK on web data remains about 1.7% below the 3DPW ground truth.
- Generalizable methods show significant domain gaps between in-the-wild and laboratory data.
Experimental setup
Datasets: WildAvatar EMDB 3DPW ZJU-Mocap RenderPeople THuman HuMMan
Benchmarks: EMDB WildAvatar ZJU-Mocap RenderPeople THuman HuMMan
Metrics: PA-MPJPE MPJPE PVE PSNR SSIM LPIPS PCK SSIOU
Baselines: SPIN VIBE PARE TRACE CLIFF TCMR HybrIK HMR2.0 NHR NeuralBody Animatable NeRF Animatable SDF
Builds on: SMPL Segment Anything HMR2.0 NeuralBody GauHuman
Closest papers by meaning
- Vid2Avatar-Pro: Authentic Avatar from Videos in the Wild via Universal Prior (CVPR 2025 · similarity 0.89)
- DreamAvatar: Text-and-Shape Guided 3D Human Avatar Generation via Diffusion Models (CVPR 2024 · similarity 0.86)
- AvatarVerse: High-Quality & Stable 3D Avatar Creation from Text and Pose (AAAI 2024 · similarity 0.86)
- HumanVid: Demystifying Training Data for Camera-controllable Human Image Animation (NeurIPS 2024 · similarity 0.85)
- GaussianAvatar: Towards Realistic Human Avatar Modeling from a Single Video via Animatable 3D Gaussians (CVPR 2024 · similarity 0.85)
- FRESA: Feedforward Reconstruction of Personalized Skinned Avatars from Few Images (CVPR 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.