Research map / 3D vision / Face and shape
ANIM: Accurate Neural Implicit Model for Human Reconstruction from a single RGB-D Image
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes ANIM, a pixel-voxel-aligned neural implicit model with a depth-supervision loss that reconstructs clothed 3D humans from a single RGB-D image, and introduces the ANIM-Real dataset for fine-tuning on consumer-grade sensor noise.
Problem
Single-view human reconstruction from RGB or RGB-D struggles with fine geometric details (face, hands, wrinkles) and depth ambiguity along the camera axis; consumer RGB-D sensors are noisy and no real-world dataset pairs them with high-quality 3D ground truth.
Approach
Combine multi-resolution pixel-aligned 2D features (from RGB and normals) with voxel-aligned 3D features from a SparseConvNet U-Net over the depth voxels, add a depth-supervision loss on input point cloud points, and use semantic-aware sampling of face and hand regions.
Key results
- On RenderPeople, ANIM reaches CD 2.075 cm, normal 0.4990, P2S 1.936, beating the best single-RGB-D baseline PIFu+VFE+SN (CD 2.242).
- On THuman2.0, ANIM reaches CD 0.913 cm and P2S 0.878, versus 0.924 and 0.880 for PIFu+VFE+SN, and far ahead of other RGB-D baselines such as NormalGAN (CD 2.830).
- Ablation: removing the depth-supervision loss raises RenderPeople CD only slightly (2.060 vs 2.075) but worsens normal error (0.5647 vs 0.4990).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use ANIM’s full RGB-D architecture | ANIM · RenderPeople and THuman2.0 | CD: 2.164 -> 2.075; 0.924 -> 0.913 | Helped |
| Use ANIM’s full RGB-D architecture | ANIM · RenderPeople and THuman2.0 | Normal: .4995 -> .4990; .2548 -> .2545 | Helped |
| Use ANIM’s full RGB-D architecture | ANIM · RenderPeople and THuman2.0 | P2S: 1.962 -> 1.936; 0.880 -> 0.878 | Helped |
| Add HR features and VFE to LR-only features (ablation) | ANIM · RenderPeople and THuman2.0 | CD: 2.653 -> 2.075; 1.998 -> 0.913 | Helped |
| Add LR features and VFE to HR-only features (ablation) | ANIM · RenderPeople and THuman2.0 | CD: 2.170 -> 2.075; 1.147 -> 0.913 | Helped |
| Add 2D feature extractors to 3D-only features (ablation) | ANIM · RenderPeople and THuman2.0 | CD: 3.021 -> 2.075; 1.961 -> 0.913 | Helped |
| Use LR image features instead of random voxel features (ablation) | ANIM · RenderPeople and THuman2.0 | CD: 2.136 -> 2.075; 1.052 -> 0.913 | Helped |
| Add surface normals to the input (ablation) | ANIM · RenderPeople and THuman2.0 | CD: 2.271 -> 2.075; 1.376 -> 0.913 | Helped |
| Add LR features (ablation) | ANIM · RenderPeople and THuman2.0 | CD: 2.453 -> 2.075; 1.653 -> 0.913 | Helped |
| Add HR features (ablation) | ANIM · RenderPeople and THuman2.0 | CD: 2.605 -> 2.075; 2.649 -> 0.913 | Helped |
| Add semantic-aware sampling (ablation) | ANIM · RenderPeople and THuman2.0 | CD: 2.636 -> 2.075; 0.993 -> 0.913 | Mixed |
| Add depth-supervision loss (ablation) | ANIM · RenderPeople and THuman2.0 | CD: 2.060 -> 2.075; 0.947 -> 0.913 | Mixed |
Surprising findings
- Removing the depth-supervision loss gives a slightly lower RenderPeople Chamfer distance (2.060 vs 2.075), so the loss mainly helps normal error and other metrics rather than CD on that set.
- Standard PIFu with depth input (PIFu+D) performs worse than RGB-only methods such as ECON and ICON on RenderPeople, showing naive depth concatenation is insufficient.
Limitations
- Applying the model directly to consumer-grade RGB-D input yields severe artifacts without fine-tuning on ANIM-Real.
- Single-view RGB-D input leaves occluded regions to shape completion, and the method is limited to static single frames.
- Future work noted as exploiting temporal information across frames for body pose and appearance fusion.
Experimental setup
Datasets: RenderPeople THuman2.0 ANIM-Real
Benchmarks: RenderPeople THuman2.0
Metrics: Chamfer distance normal reprojection error point-to-surface distance (P2S)
Baselines: IF-Net PaMIR PIFuHD ICON PHORHUM ECON SuRS PIFu+D NormalGAN OcPlanes
Builds on: PIFu PaMIR IF-Net SparseConvNet
Closest papers by meaning
- PIFuHD: Multi-Level Pixel-Aligned Implicit Function for High-Resolution 3D Human Digitization (CVPR 2020 · similarity 0.87)
- HISR: Hybrid Implicit Surface Representation for Photorealistic 3D Human Reconstruction (AAAI 2024 · similarity 0.87)
- SiTH: Single-view Textured Human Reconstruction with Image-Conditioned Diffusion (CVPR 2024 · similarity 0.86)
- Relightable and Animatable Neural Avatar from Sparse-View Video (CVPR 2024 · similarity 0.85)
- PSHuman: Photorealistic Single-image 3D Human Reconstruction using Cross-Scale Multiview Diffusion and Explicit Remeshing (CVPR 2025 · similarity 0.85)
- DiffHuman: Probabilistic Photorealistic 3D Reconstruction of Humans (CVPR 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.