Research map / 3D vision / Depth and stereo
NeISF: Neural Incident Stokes Field for Geometry and Material Estimation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes NeISF, a polarization-aware multi-view inverse rendering method that implicitly models multi-bounce incident Stokes fields with MLPs plus a differentiable polarimetric renderer to jointly estimate geometry, material and lighting.
Problem
Multi-view inverse rendering struggles with multi-bounce effects such as inter-reflections under single-bounce assumptions, and existing polarization-based NeRF inverse rendering cannot estimate BRDF parameters and assumes unpolarized incident light.
Approach
Implicitly record already-rotated incident Stokes vectors for diffuse and specular components separately using coordinate MLPs (incident Stokes fields), combined with a physically-based differentiable polarimetric renderer (Baek pBRDF) and VolSDF geometry, avoiding explicit reference-frame rotation computation.
Key results
- On the synthetic Bunny scene, NeISF achieves normal MAE of 1.727° versus 7.769° for PANDORA and 4.481° for NeILF++, and mixed PSNR of 36.81 versus 24.62 for PANDORA.
- On the synthetic Teapot scene, NeISF achieves normal MAE of 2.541° and mixed PSNR of 31.32, outperforming Ours-no-pol (3.894°, 30.23) and PANDORA (6.722°, 22.47).
- Qualitatively, NeISF reconstructs inter-reflections and clean normals on real data where PANDORA, VolSDF and NeILF++ confuse albedo patterns with geometry.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add polarization cues (ablation) | NeISF · Bunny | normal MAE: 3.295° -> 1.727° | Helped |
| Add polarization cues (ablation) | NeISF · Bunny | Mixed PSNR: 34.45 -> 36.81 | Helped |
| Add polarization cues (ablation) | NeISF · Bunny | Specular PSNR: 26.91 -> 27.68 | Helped |
| Add polarization cues (ablation) | NeISF · Bunny | Diffuse PSNR: 32.93 -> 35.31 | Helped |
| Add polarization cues (ablation) | NeISF · Bunny | Roughness SI-L1: .0244 -> .0149 | Helped |
| Add polarization cues (ablation) | NeISF · Bunny | Albedo SI-L1: .0396 -> .0372 | Helped |
| Add polarization cues (ablation) | NeISF · Teapot | normal MAE: 3.894° -> 2.541° | Helped |
| Add polarization cues (ablation) | NeISF · Teapot | Mixed PSNR: 30.23 -> 31.32 | Helped |
| Add polarization cues (ablation) | NeISF · Teapot | Specular PSNR: 21.15 -> 22.50 | Helped |
| Add polarization cues (ablation) | NeISF · Teapot | Diffuse PSNR: 30.17 -> 31.97 | Helped |
| Add polarization cues (ablation) | NeISF · Teapot | Roughness SI-L1: .0223 -> .0172 | Helped |
| Add polarization cues (ablation) | NeISF · Teapot | Albedo SI-L1: .0720 -> .0712 | Helped |
Surprising findings
- Texture patterns (albedo) are misinterpreted as geometry by VolSDF, NeILF++ and PANDORA, which NeISF avoids on real data.
Limitations
- Estimated implicit lighting cannot be used in conventional renderers.
- Only opaque dielectric objects are supported; metals and translucent objects are excluded.
- Real-world Stokes vectors are noisy due to misaligned polarization sensor pixels, limiting high-frequency detail; only indoor object-level scenes studied.
Experimental setup
Datasets: synthetic polarimetric HDR dataset rendered with Mitsuba 3.0 and Baek pBRDF real-world HDR polarimetric dataset captured with FLIR polarization camera
Benchmarks: own synthetic polarimetric HDR dataset own real-world polarimetric HDR dataset
Metrics: mean angular error PSNR scale-invariant L1
Baselines: PANDORA NeILF++ VolSDF Ours-no-pol
Builds on: NeILF VolSDF Baek pBRDF NeRF Neural Radiosity
Closest papers by meaning
- NeISF++: Neural Incident Stokes Field for Polarized Inverse Rendering of Conductors and Dielectrics (CVPR 2025 · similarity 0.93)
- Inverse Rendering of Glossy Objects via the Neural Plenoptic Function and Radiance Fields (CVPR 2024 · similarity 0.90)
- Glossy Object Reconstruction with Cost-effective Polarized Acquisition (CVPR 2025 · similarity 0.88)
- Factored-NeuS: Reconstructing Surfaces, Illumination, and Materials of Possibly Glossy Objects (CVPR 2025 · similarity 0.87)
- PBR-NeRF: Inverse Rendering with Physics-Based Neural Fields (CVPR 2025 · similarity 0.87)
- VMINer: Versatile Multi-view Inverse Rendering with Near- and Far-field Light Sources (CVPR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.