Research map / 3D vision / Face and shape
NECA: Neural Customizable Human Avatar
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces NECA, a neural human avatar method that learns disentangled SDF, albedo, shadow and environment-lighting fields from monocular or sparse-view videos using dual (canonical tri-plane + SMPL surface) features for fully customizable rendering.
Problem
Existing neural human avatars support only limited editing (pose, shape, or lighting/texture), lacking full customization of pose, shadow, shape, lighting and texture from monocular or sparse-view videos.
Approach
Represent humans in complementary dual spaces: a pose-aware tri-plane feature in canonical space (CP-factorized, pose-dictionary weighted) and a subject-level feature on the SMPL surface (UV latent codes plus tangent-space local coordinates); decode these into disentangled SDF, albedo and shadow fields plus a learnable spherical light probe, rendered with a data-driven rendering equation.
Key results
- On ZJU-MoCap novel pose synthesis, NECA reaches PSNR 25.0, SSIM 0.914, LPIPS 0.091, outperforming ARAH (24.5/0.909/0.109) and PoseVocab (24.3/0.905/0.095).
- Novel view synthesis PSNR 28.5 versus 28.4 for ARAH, with LPIPS 0.067 best among compared methods.
- Qualitative relighting under novel pose shows more natural results than Relighting4D, which fails under dramatic movement.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use NECA instead of ARAH | ZJU-MoCap novel view synthesis · ZJU-MoCap novel view | PSNR: 28.4 -> 28.5 | Helped |
| Use NECA instead of ARAH | ZJU-MoCap novel view synthesis · ZJU-MoCap novel view | SSIM: 0.947 -> 0.946 | Hurt |
| Use NECA instead of PoseVocab | ZJU-MoCap novel view synthesis · ZJU-MoCap novel view | LPIPS: 0.068 -> 0.067 | Helped |
| Use NECA instead of ARAH | ZJU-MoCap novel pose synthesis · ZJU-MoCap novel pose | PSNR: 24.5 -> 25.0 | Helped |
| Use NECA instead of ARAH | ZJU-MoCap novel pose synthesis · ZJU-MoCap novel pose | SSIM: 0.909 -> 0.914 | Helped |
| Use NECA instead of PoseVocab | ZJU-MoCap novel pose synthesis · ZJU-MoCap novel pose | LPIPS: 0.095 -> 0.091 | Helped |
| Add normal regularization Ln (ablation) | NECA · ZJU-MoCap novel view and pose (4 views) | PSNR: 26.6 -> 26.8 | Helped |
| Add normal regularization Ln (ablation) | NECA · ZJU-MoCap novel view and pose (4 views) | SSIM: 0.928 -> 0.930 | Helped |
| Add normal regularization Ln (ablation) | NECA · ZJU-MoCap novel view and pose (4 views) | LPIPS: 0.084 -> 0.079 | Helped |
| Add pose-aware feature (ablation) | NECA · ZJU-MoCap novel view and pose (4 views) | PSNR: 26.6 -> 26.8 | Helped |
| Add pose-aware feature (ablation) | NECA · ZJU-MoCap novel view and pose (4 views) | LPIPS: 0.078 -> 0.079 | No clear effect |
| Add subject-level feature (ablation) | NECA · ZJU-MoCap novel view and pose (4 views) | PSNR: 26.6 -> 26.8 | Helped |
Limitations
- Performance sensitive to estimated SMPL parameters since subject-level feature relies on nearest SMPL point.
- Shadows under complex novel poses may be erroneous because visibility is not explicitly modeled.
- Each human must be optimized separately, like vanilla NeRF.
Experimental setup
Datasets: ZJU-MoCap NeuMan DeepCap DynaCap
Benchmarks: ZJU-MoCap NeuMan DeepCap DynaCap
Metrics: PSNR SSIM LPIPS
Baselines: Neural Body Animatable NeRF Dual-Space NeRF ARAH PoseVocab Relighting4D
Builds on: Neural Body Animatable NeRF ARAH PoseVocab Relighting4D NeRF
Closest papers by meaning
- Relightable and Animatable Neural Avatars from Videos (AAAI 2024 · similarity 0.89)
- IntrinsicAvatar: Physically Based Inverse Rendering of Dynamic Humans from Monocular Videos via Explicit Ray Tracing (CVPR 2024 · similarity 0.88)
- RAM-Avatar: Real-time Photo-Realistic Avatar from Monocular Videos with Full-body Control (CVPR 2024 · similarity 0.87)
- Relightable and Animatable Neural Avatar from Sparse-View Video (CVPR 2024 · similarity 0.87)
- Vid2Avatar-Pro: Authentic Avatar from Videos in the Wild via Universal Prior (CVPR 2025 · similarity 0.87)
- 3DGS-Avatar: Animatable Avatars via Deformable 3D Gaussian Splatting (CVPR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.