Research map / 3D vision / Face and shape
IntrinsicAvatar: Physically Based Inverse Rendering of Dynamic Humans from Monocular Videos via Explicit Ray Tracing
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Recovers geometry, albedo, material and lighting of clothed avatars from monocular videos by combining an articulated iNGP-SDF neural field with Monte-Carlo volumetric scattering and explicit secondary ray tracing in canonical space.
Problem
Physically based inverse rendering of clothed human avatars (geometry, albedo, material, lighting) from monocular videos, where prior methods bake intrinsic properties together or approximate secondary shading (shadows, indirect light) with learned MLPs that generalize poorly to novel poses.
Approach
Model the clothed human as a volumetric scattering medium with an SDF-derived density, and trace secondary rays explicitly in canonical space using a zero-crossing iso-surface search followed by importance-weighted accumulation, so visibility is computed exactly and generalizes to novel poses without learned visibility MLPs.
Key results
- On RANA, the method reduces normal error to 9.96 degrees versus 27.38 for R4D* (64% reduction) and 42.69 for R4D.
- Albedo PSNR of 22.83 vs 18.23 for R4D* on RANA training poses.
- Novel-pose relighting SSIM of 0.8722 vs 0.8370 for R4D*; training takes about 4 hours on a single RTX 3090.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use IntrinsicAvatar instead of R4D* | Clothed human avatar reconstruction · RANA | Albedo PSNR: 18.23 -> 22.83 | Helped |
| Use IntrinsicAvatar instead of R4D* | Clothed human avatar reconstruction · RANA | Albedo SSIM: 0.8254 -> 0.8816 | Helped |
| Use IntrinsicAvatar instead of R4D* | Clothed human avatar reconstruction · RANA | Albedo LPIPS: 0.2043 -> 0.1617 | Helped |
| Use IntrinsicAvatar instead of R4D* | Clothed human avatar reconstruction · RANA | Normal error: 27.38 ° -> 9.96 ° | Helped |
| Use IntrinsicAvatar instead of R4D* | Clothed human avatar reconstruction · RANA novel poses | Relighting PSNR: 16.62 -> 18.18 | Helped |
| Use IntrinsicAvatar instead of R4D* | Clothed human avatar reconstruction · RANA novel poses | Relighting SSIM: 0.8370 -> 0.8722 | Helped |
| Use IntrinsicAvatar instead of R4D* | Clothed human avatar reconstruction · RANA novel poses | Relighting LPIPS: 0.1726 -> 0.1279 | Helped |
| Replace volumetric scattering with rendered depth and surface scattering (ablation) | IntrinsicAvatar · RANA subject 01 | Albedo PSNR: 24.1 -> 24.05 | Hurt |
| Replace volumetric scattering with rendered depth and surface scattering (ablation) | IntrinsicAvatar · RANA subject 01 | Albedo SSIM: 0.8679 -> 0.8671 | Hurt |
| Replace volumetric scattering with rendered depth and surface scattering (ablation) | IntrinsicAvatar · RANA subject 01 | Normal error: 12.05° -> 11.84° | Helped |
| Add iso-surface search for secondary ray tracing (ablation) | IntrinsicAvatar · RANA subject 01 | Albedo PSNR: 23.75 -> 24.05 | Helped |
| Add iso-surface search for secondary ray tracing (ablation) | IntrinsicAvatar · RANA subject 01 | Albedo SSIM: 0.8651 -> 0.8671 | Helped |
Surprising findings
- Rendered-depth surface scattering (as in TensoIR) produces discontinuities at boundaries and edges, while volumetric scattering avoids them.
Limitations
- Relies on SMPL-based articulation and pose estimates, which may not align perfectly with images (affects PSNR on RANA)
- Secondary ray tracing and volumetric scattering are costly relative to surface rendering
Experimental setup
Datasets: RANA PeopleSnapshot SyntheticHuman-Relit
Benchmarks: RANA PeopleSnapshot SyntheticHuman-Relit
Metrics: albedo PSNR albedo SSIM albedo LPIPS normal error (degrees) relighting PSNR relighting SSIM relighting LPIPS
Baselines: Relighting4D R4D* TensoIR RANA
Builds on: NeRF VolSDF instant NGP Fast-SNARF NeuS TensoIR
Closest papers by meaning
- Vid2Avatar-Pro: Authentic Avatar from Videos in the Wild via Universal Prior (CVPR 2025 · similarity 0.89)
- Human Gaussian Splatting: Real-time Rendering of Animatable Avatars (CVPR 2024 · similarity 0.88)
- 3DGS-Avatar: Animatable Avatars via Deformable 3D Gaussian Splatting (CVPR 2024 · similarity 0.88)
- Relightable and Animatable Neural Avatars from Videos (AAAI 2024 · similarity 0.88)
- NECA: Neural Customizable Human Avatar (CVPR 2024 · similarity 0.88)
- Relightable and Animatable Neural Avatar from Sparse-View Video (CVPR 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.