Research map / 3D vision / Face and shape
Eclipse: Disambiguating Illumination and Materials using Unintended Shadows
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Jointly recovers spatially-varying materials, environment illumination, and unseen occluder shapes from multi-view images by exploiting unintended shadows from the photographer, using differentiable Monte Carlo ray tracing.
Problem
Recovering object materials and surrounding illumination from images is ill-posed and ill-conditioned, especially for diffuse objects, since diffuse BRDFs low-pass filter lighting and diffuse-under-sharp-light looks like shiny-under-blurry-light.
Approach
Treat unintended shadows cast by unobserved moving occluders (e.g., the camera operator) as a signal: model each occluder as a spherical-harmonic binary-like mask per view and jointly optimize it with BRDF (MLP), pyramid environment map, via differentiable Monte Carlo ray tracing.
Key results
- Including unintended occluders reduces illumination relative MSE versus no-occluder baseline across illumination frequencies, roughnesses, and mask sizes (Figure 9).
- On synthetic diffuse scenes, full model achieves RMSE 0.038 (chair), 0.035 (mannequin), 0.011 (potatoes), 0.046 (toad) versus 0.831/0.654/0.997/0.943 for direct spherical harmonics occluder parameterization (Table 1).
- Recovers plausible illumination and albedo on real Stanford-ORB salt004 scene despite not being designed for the task (RMSE 0.087 vs 0.035 for pictured case).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| model unintended occluders (ablation) | Eclipse inverse rendering · Stanford-ORB salt004 | albedo PSNR: 14.4 dB -> 19.3 dB | Helped |
| model unintended occluders (ablation) | Eclipse inverse rendering · Stanford-ORB salt004 | illumination RMSE: 0.087 -> 0.035 | Helped |
| use spherical harmonics for occluder masks instead of directly optimizing mask values (ablation) | Eclipse · chair, mannequin, potatoes, toad | RMSE: 0.831 -> 0.038; 0.654 -> 0.035; 0.997 -> 0.011; 0.943 -> 0.046 | Helped |
| use an image-pyramid environment map instead of directly optimizing environment-map values (ablation) | Eclipse · chair, mannequin, potatoes, toad | RMSE: 0.042 -> 0.038; 0.071 -> 0.035; 0.015 -> 0.011; 0.073 -> 0.046 | Mixed |
| include unintended moving-occluder shadows | Eclipse illumination recovery · varying illumination frequency content, material roughness, and mask sizes | relative MSE | Helped |
| increase unseen occluder size | Eclipse illumination recovery · varying mask sizes | relative MSE | Helped |
| use object self-occlusions without external occluders (ablation) | Eclipse illumination recovery · diffuse objects | relative MSE | Hurt |
Surprising findings
- Larger occluders that block more light improve illumination recovery quality.
- Plausible illumination and albedo can be recovered even when object geometry is unknown, using NeRF-estimated proxy geometry.
Limitations
- Neglects global illumination and lower-order effects for efficiency.
- Recovered occluder masks are less accurate in dark illumination regions.
- Real-data results are imperfect and the method was not designed for the Stanford-ORB capture setup.
Experimental setup
Datasets: Stanford-ORB synthetic rendered diffuse and shiny objects
Benchmarks: Stanford-ORB salt004 scene synthetic diffuse objects (chair, mannequin, potatoes, toad)
Metrics: RMSE PSNR relative MSE
Baselines: Swedish et al. (no occluders) Direct spherical harmonics occluder parameterization Direct environment map parameterization
Builds on: Swedish et al. (shadow-based lighting estimation) Park et al. (RGBD environment map recovery) Verbin et al. (orientation loss NeRF) NeRF GGX microfacet model
Closest papers by meaning
- Factored-NeuS: Reconstructing Surfaces, Illumination, and Materials of Possibly Glossy Objects (CVPR 2025 · similarity 0.90)
- Inverse Rendering of Glossy Objects via the Neural Plenoptic Function and Radiance Fields (CVPR 2024 · similarity 0.89)
- Multi-times Monte Carlo Rendering for Inter-reflection Reconstruction (NeurIPS 2024 · similarity 0.89)
- Generative Multiview Relighting for 3D Reconstruction under Extreme Illumination Variation (CVPR 2025 · similarity 0.89)
- Material Palette: Extraction of Materials from a Single Image (CVPR 2024 · similarity 0.89)
- MaterialRefGS: Reflective Gaussian Splatting with Multi-view Consistent Material Inference (NeurIPS 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.