Research map / 3D vision / Gaussian splatting
Gaussian Shadow Casting for Neural Characters
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Adds an analytic anisotropic-Gaussian density proxy for closed-form shadow ray transmittance to a deferred Lambertian shading layer on top of a neural body model (DANBO), enabling shadow-aware reconstruction and relighting of neural characters from outdoor video.
Problem
Neural character models reconstructed from video bake global cast shadows into albedo, causing artifacts in novel poses and when relighting; explicit secondary ray casting through a NeRF is too costly.
Approach
Fit a skeleton-attached sum of anisotropic 3D Gaussians to the NeRF density and integrate it along shadow rays in closed form (1D Gaussian reduction plus erf CDF), avoiding sampling, affine projection, and sorting; combine with deferred Lambertian shading so only one shadow ray per pixel is cast.
Key results
- On the synthetic novel-pose test set, GSC reaches 22.04 PSNR versus 17.52 for DANBO and 17.57 for NPC, and averages 19.30 PSNR across synthetic and real sequences versus 17.26 (DANBO).
- Casting a Gaussian shadow adds only about 0.3 s per render (DANBO+DS 17.13 s to 17.47 s), versus 21.4 s for NeRF-based shadow casting.
- Light direction is recovered without user input, with 9.9 degree error on RANA's monocular data and 1.36 degree error when initialized from behind on the synthetic sequence.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add Gaussian shadow casting and diffuse shading | DANBO · Synthetic novel pose | PSNR: 17.52 -> 22.04 | Helped |
| add Gaussian shadow casting and diffuse shading | DANBO · Real S1 novel pose | PSNR: 16.57 -> 17.57 | Helped |
| add Gaussian shadow casting and diffuse shading | DANBO · Real S2 novel pose | PSNR: 17.69 -> 18.29 | Helped |
| add Gaussian shadow casting and diffuse shading | DANBO · Average novel pose | PSNR: 17.26 -> 19.30 | Helped |
| add Gaussian shadow casting and diffuse shading | DANBO · Synthetic self-casting shadow novel pose | PSNR: 17.78 -> 22.13 | Helped |
| add Gaussian shadow casting and diffuse shading | DANBO · Real S1 self-casting shadow novel pose | PSNR: 15.11 -> 16.88 | Helped |
| add Gaussian shadow casting and diffuse shading | DANBO · Real S2 self-casting shadow novel pose | PSNR: 16.55 -> 17.40 | Helped |
| add Gaussian shadow casting and diffuse shading | DANBO · Average self-casting shadow novel pose | PSNR: 16.48 -> 18.81 | Helped |
| add diffuse shading (ablation) | Ours · Novel pose (Synthetic Sequence) | PSNR: 21.22 -> 22.30 | Helped |
| add diffuse shading (ablation) | Ours · Novel view (Synthetic Sequence) | PSNR: 25.08 -> 27.32 | Helped |
| allow gradients to propagate through surface normals during diffuse shading (ablation) | Ours · Novel pose (Synthetic Sequence) | PSNR: 20.43 -> 22.30 | Helped |
| allow gradients to propagate through surface normals during diffuse shading (ablation) | Ours · Novel view (Synthetic Sequence) | PSNR: 26.18 -> 27.32 | Helped |
Surprising findings
- SSIM and LPIPS favor baselines (DANBO, NPC) despite GSC's higher PSNR, which the authors attribute to brightness/contrast normalization in those metrics.
- Providing ground-truth light direction did not improve results and may have hindered optimization.
Limitations
- Smooth low-resolution Gaussians cannot represent fine shadow details such as fingers or nose.
- Disentangling albedo and shading yields slightly lower reconstruction metrics when shading effects are minimal.
- Relighting4D and RANA could not be fully compared because their code was unavailable; RANA comparison limited to albedo estimates.
Experimental setup
Datasets: RANA synthetic dataset Synthetic Mixamo/Cycles sequence (own, from VITUS 3D body scan) Outdoor sunlight dataset (own capture, 2 sequences)
Benchmarks: RANA synthetic dataset Outdoor sunlight dataset (novel view and novel pose)
Metrics: PSNR SSIM LPIPS render time
Baselines: DANBO NPC Relighting4D RANA NeRF shadow casting (NeRFSC)
Builds on: DANBO Rhodin et al. Gaussian visibility (ray-Gaussian occlusion) Gaussian Splatting NeRF
Closest papers by meaning
- RNG: Relightable Neural Gaussians (CVPR 2025 · similarity 0.90)
- Human Gaussian Splatting: Real-time Rendering of Animatable Avatars (CVPR 2024 · similarity 0.87)
- MiraGe: Editable 2D Images using Gaussian Splatting (ICML 2025 · similarity 0.87)
- MaskGaussian: Adaptive 3D Gaussian Representation from Probabilistic Masks (CVPR 2025 · similarity 0.87)
- Compact 3D Gaussian Representation for Radiance Field (CVPR 2024 · similarity 0.86)
- Structure Consistent Gaussian Splatting with Matching Prior for Few-shot Novel View Synthesis (NeurIPS 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.