Research map / 3D vision / Depth and stereo
RNb-NeuS: Reflectance and Normal-based Multi-View 3D Reconstruction
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a pixel-wise joint re-parameterization of PS reflectance and normals into Lambertian radiance vectors, integrated into NeuS for single-objective multi-view photometric stereo reconstruction on DiLiGenT-MV.
Problem
Multi-view photometric stereo (MVPS) fuses per-view reflectance and normal maps with multi-view geometry, but existing methods use multi-objective optimization that can lose fine geometric details.
Approach
Re-parameterize each pixel's reflectance and normal as a vector of radiances simulated under three arbitrary Lambertian illuminations (v = r L n), so both inputs become homogeneous intensities matched by a NeuS-style volume renderer with a single L1 data objective plus eikonal regularization.
Key results
- Chamfer distance averaged 0.23 mm on DiLiGenT-MV, 17.4% better than the second-best MVPSNet (0.27).
- Normal MAE averaged 4.95 degrees, similar to Kaya23 (5.14) and better than SDM-UniPS single-view (7.12).
- On high curvature and low visibility areas, Chamfer distance rose only 4% and 13% respectively, versus 36%-96% for competing methods.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add reflectance maps to the normal-based reconstruction (ablation) | RNb-NeuS · DiLiGenT-MV average | Chamfer distance: 0.26 -> 0.23 | Helped |
| use pixel-wise optimal lighting triplets (ablation) | RNb-NeuS · DiLiGenT-MV average | Chamfer distance: 0.24 -> 0.23 | Helped |
| discard less reliable normal and reflectance inputs using uncertainty (ablation) | RNb-NeuS · DiLiGenT-MV average | Chamfer distance: 0.23 -> 0.23 | No clear effect |
Surprising findings
- Using reflectance was the factor with the largest impact on reconstruction accuracy in the ablation.
Limitations
- Relies heavily on quality of PS normal maps; SDM-UniPS occasionally yields corrupted normals.
- Computation time of 8 to 16 hours per object is high.
Experimental setup
Datasets: DiLiGenT-MV
Benchmarks: DiLiGenT-MV
Metrics: F-score Chamfer distance mean angular error
Baselines: Park16 Li19 Kaya22 PS-NeRF Kaya23 MVPSNet SDM-UniPS
Builds on: NeuS SDM-UniPS PS-NeRF Kaya et al. 2023 MVPS
Closest papers by meaning
- MVCPS-NeuS: Multi-view Constrained Photometric Stereo for Neural Surface Reconstruction (CVPR 2024 · similarity 0.90)
- Deep 3D Capture: Geometry and Reflectance From Sparse Multi-View Images (CVPR 2020 · similarity 0.88)
- SuperNormal: Neural Surface Reconstruction via Multi-View Normal Integration (CVPR 2024 · similarity 0.87)
- GNeRP: Gaussian-guided Neural Reconstruction of Reflective Objects with Noisy Polarization Priors (ICLR 2024 · similarity 0.86)
- Sparse Views Near Light: A Practical Paradigm for Uncalibrated Point-light Photometric Stereo (CVPR 2024 · similarity 0.86)
- Normal-NeRF: Ambiguity-Robust Normal Estimation for Highly Reflective Scenes (AAAI 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.