Research map / 3D vision / Depth and stereo
NeRSP: Neural 3D Reconstruction for Reflective Objects with Sparse Polarized Images
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes NeRSP, a neural SDF-based 3D reconstruction method that fuses photometric (Stokes/pBRDF) and geometric (multiview AoP azimuth) polarization cues to reconstruct reflective objects from as few as 6 views, and releases the RMVP3D real dataset.
Problem
Multiview 3D reconstruction of reflective (specular) surfaces from sparse input views, where view-dependent reflectance breaks photometric consistency and sparse views lack correspondences.
Approach
Jointly optimizes an implicit SDF surface using a polarimetric photometric loss (rendered vs. observed Stokes vectors via pBRDF) and a multiview azimuth geometric loss (linear constraint from AoP maps across views), so the two cues together narrow the shape solution space under sparse views.
Key results
- On synthetic SMVP3D with 6 views, NeRSP achieves the lowest Chamfer distance on all 5 objects (e.g., HEDGEHOG 3.43 vs. 5.37 for MVAS).
- On real RMVP3D, NeRSP reports average Chamfer distance 6.13 vs. 7.48 for NeRO and 7.86 for MVAS.
- Surface normal mean angular error is lowest on average on synthetic data (e.g., DRAGON 18.03 vs. 24.29 for MVAS).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add the photometric loss Lp (ablation) | NeRSP · SMVP3D DRAGON | Chamfer distance: 5.17 -> 4.77 | Helped |
| add the geometric loss Lg (ablation) | NeRSP · SMVP3D DRAGON | Chamfer distance: 6.09 -> 4.77 | Helped |
| add the photometric loss Lp (ablation) | NeRSP · SMVP3D DRAGON | MAE: 17.49 -> 14.57 | Helped |
| add the geometric loss Lg (ablation) | NeRSP · SMVP3D DRAGON | MAE: 23.78 -> 14.57 | Helped |
Surprising findings
- MVAS has the second-smallest Chamfer distance on some objects yet over 20 degrees mean angular error, indicating smooth shapes despite low geometric error.
Limitations
- Inter-reflections and polarized environment lighting are not modeled.
- Shape-radiance ambiguity remains under very sparse views for some objects.
- Real-data ground truth alignment relies on PANDORA estimate plus ICP.
Experimental setup
Datasets: SMVP3D RMVP3D PANDORA dataset
Benchmarks: SMVP3D RMVP3D PANDORA dataset
Metrics: Chamfer distance mean angular error
Baselines: NeRO S-VolSDF MVAS PANDORA
Builds on: NeuS Ref-NeRF PANDORA MVAS pBRDF
Closest papers by meaning
- GNeRP: Gaussian-guided Neural Reconstruction of Reflective Objects with Noisy Polarization Priors (ICLR 2024 · similarity 0.91)
- High-Fidelity Polarimetric Implicit 3D Reconstruction with View-Dependent Physical Representation (AAAI 2025 · similarity 0.88)
- UniSDF: Unifying Neural Representations for High-Fidelity 3D Reconstruction of Complex Scenes with Reflections (NeurIPS 2024 · similarity 0.88)
- NeuSurf: On-Surface Priors for Neural Surface Reconstruction from Sparse Input Views (AAAI 2024 · similarity 0.88)
- Normal-NeRF: Ambiguity-Robust Normal Estimation for Highly Reflective Scenes (AAAI 2025 · similarity 0.87)
- SC-NeuS: Consistent Neural Surface Reconstruction from Sparse and Noisy Views (AAAI 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.