Research map / 3D vision / Face and shape
NeISF++: Neural Incident Stokes Field for Polarized Inverse Rendering of Conductors and Dielectrics
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Extends the NeISF polarized inverse rendering pipeline to conductors and dielectrics via a general pBRDF with a conductor/dielectric indicator and complex Fresnel term, plus DoLP-based SDF initialization, and releases new synthetic and real HDR polarized datasets.
Problem
Polarized inverse rendering methods only support dielectrics, causing large geometry and material errors on conductors, which are glossy and produce strong specular reflections that are hard to reconstruct.
Approach
A general pBRDF that uses a binary indicator to disable diffuse polarization for conductors and a Fresnel reflection term supporting complex refractive indices (estimated per point), combined with a geometry initialization that trains an SDF on DoLP images, which are invariant to light intensity and robust to specular highlights.
Key results
- On the synthetic dataset, NeISF++ achieves mean normal angular error of 6.487° (Stanford scan) and 1.789° (Helmet), versus 14.022° and 10.303° for NeISF.
- Material reconstruction MAE on Stanford scan: roughness 0.0706 and albedo 0.0468 for NeISF++, versus 0.2425 and 0.2954 for NeISF.
- The ablation Ours-a (VolSDF initialization instead of DoLP) yields higher normal error (6.480° vs 6.487° on Stanford scan is nearly equal, but 2.400° vs 1.789° on Helmet), showing DoLP initialization helps on the Helmet scene.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use DoLP-based geometry initialization instead of original VolSDF initialization (ablation) | NeISF++ · Synthetic dataset, Stanford scan and Helmet | mean angular error: 6.480◦ -> 6.487◦; 2.400◦ -> 1.789◦ | Mixed |
| Use DoLP-based geometry initialization instead of original VolSDF initialization (ablation) | NeISF++ · Synthetic dataset, Stanford scan | roughness MAE: .0821 -> .0706 | Helped |
| Use DoLP-based geometry initialization instead of original VolSDF initialization (ablation) | NeISF++ · Synthetic dataset, Stanford scan | albedo MAE: .1289 -> .0468 | Helped |
| Use DoLP-based geometry initialization instead of original VolSDF initialization (ablation) | NeISF++ · Synthetic dataset, Stanford scan | ior (real) MAE: .0722 -> .0685 | Helped |
| Use DoLP-based geometry initialization instead of original VolSDF initialization (ablation) | NeISF++ · Synthetic dataset, Stanford scan | ior (imag.) MAE: .5107 -> .4300 | Helped |
| Use DoLP-based geometry initialization instead of original VolSDF initialization (ablation) | NeISF++ · Synthetic dataset, Helmet | roughness MAE: .0199 -> .0161 | Helped |
| Use DoLP-based geometry initialization instead of original VolSDF initialization (ablation) | NeISF++ · Synthetic dataset, Helmet | albedo MAE: .0716 -> .0615 | Helped |
| Use DoLP-based geometry initialization instead of original VolSDF initialization (ablation) | NeISF++ · Synthetic dataset, Helmet | ior (real) MAE: .1937 -> .0717 | Helped |
| Use DoLP-based geometry initialization instead of original VolSDF initialization (ablation) | NeISF++ · Synthetic dataset, Helmet | ior (imag.) MAE: 1.327 -> .6526 | Helped |
| Use a general pBRDF that supports conductors and dielectrics | NeISF++ · Synthetic dataset, Stanford scan and Helmet | roughness MAE: .2425 -> .0706; .2075 -> .0161 | Helped |
| Use a general pBRDF that supports conductors and dielectrics | NeISF++ · Synthetic dataset, Stanford scan and Helmet | albedo MAE: .2954 -> .0468; .4467 -> .0615 | Helped |
| Use NeISF++ instead of NeISF | NeISF++ · Synthetic dataset, Stanford scan and Helmet | mean angular error: 14.022◦ -> 6.487◦; 10.303◦ -> 1.789◦ | Helped |
Surprising findings
- On the Stanford scan, Ours-a has a marginally lower normal error than the full method (6.480° vs 6.487°).
Limitations
- Conductor-dielectric masks must be supplied by the user; manual creation is labor-intensive.
- Dielectric IOR is fixed at 1.5 and only conductor IOR is estimated.
- Evaluation on real data is qualitative only due to lack of ground truth.
Experimental setup
Datasets: Synthetic polarized HDR dataset rendered with Mitsuba 3 Real polarized HDR dataset captured with FLIR BFS-U3-51S5PC-C polarization camera
Benchmarks: Proposed synthetic dataset (Stanford scan, Helmet) Proposed real dataset (qualitative)
Metrics: mean angular error (surface normal) mean absolute error (roughness, albedo, ior real/imag)
Baselines: NeISF PANDORA NeRO VolSDF VolSDF-DoLP (ablation) Ours-a (ablation)
Builds on: NeISF Baek pBRDF VolSDF Disney BRDF PANDORA
Closest papers by meaning
- NeISF: Neural Incident Stokes Field for Geometry and Material Estimation (CVPR 2024 · similarity 0.93)
- Inverse Rendering of Glossy Objects via the Neural Plenoptic Function and Radiance Fields (CVPR 2024 · similarity 0.89)
- Glossy Object Reconstruction with Cost-effective Polarized Acquisition (CVPR 2025 · similarity 0.86)
- Factored-NeuS: Reconstructing Surfaces, Illumination, and Materials of Possibly Glossy Objects (CVPR 2025 · similarity 0.86)
- NeRSP: Neural 3D Reconstruction for Reflective Objects with Sparse Polarized Images (CVPR 2024 · similarity 0.86)
- Multi-times Monte Carlo Rendering for Inter-reflection Reconstruction (NeurIPS 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.