Research map / 3D vision / Depth and stereo
MVCPS-NeuS: Multi-view Constrained Photometric Stereo for Neural Surface Reconstruction
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes multi-view constrained photometric stereo (light moves with camera), factorizes multi-view observations with HO-GSVD into normals with one global ambiguity, and resolves it inside a NeuS-based neural surface reconstruction with variance-based confidence weighting.
Problem
Multi-view photometric stereo normally needs calibrated lights per view; uncalibrated per-view UPS leaves a separate linear ambiguity in each view's normals, which is hard to resolve with sparse views and few light sources.
Approach
When the light directions are unknown but move rigidly with the camera, the light directions are shared across views, so HO-GSVD reduces the per-view normal ambiguities to one global linear ambiguity that is jointly optimized with a NeuS SDF using a normal consistency loss and a variance-based per-pixel confidence weighting.
Key results
- With four views and three light sources, the proposed method achieves the lowest Chamfer distance on most scenes in Table 1, e.g., 2.28 on BLOBBY versus 7.61 for MonoSDF and 30.9 for PS-NeRF.
- HO-GSVD disambiguation yields lower error than per-view SVD ambiguity: 0.13 vs 0.62 MAE (UPS) with 4 views on the BUNNY ground-truth-normal ablation.
- Confidence weighting lowers the influence of shadowed pixels, as shown qualitatively on the BEAR ear.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use MVCPS-NeuS instead of the strongest listed baseline | BLOBBY reconstruction · BLOBBY | CD: 7.05 -> 2.28 | Helped |
| Use MVCPS-NeuS instead of the strongest listed baseline | BUNNY reconstruction · BUNNY | CD: 7.61 -> 5.38 | Helped |
| Use MVCPS-NeuS instead of the strongest listed baseline | BEAR reconstruction · BEAR | CD: 1.7 -> 1.30 | Helped |
| Use MVCPS-NeuS instead of the strongest listed baseline | BUDDHA reconstruction · BUDDHA | MAE (SDF): 22.5 -> 19.3 | Helped |
| Use MVCPS-NeuS instead of the strongest listed baseline | RABBIT reconstruction · RABBIT | CD: 2.20 -> 1.65 | Helped |
| Use MVCPS-NeuS instead of the strongest listed baseline | CUBE reconstruction · CUBE | CD: 0.90 -> 0.85 | Helped |
| Use MVCPS-NeuS instead of the strongest listed baseline | FROG reconstruction · FROG | CD: 19.3 -> 14.8 | Helped |
| Replace per-view SVD-based UPS with HO-GSVD (ablation) | MVCPS-NeuS · BUNNY, 4 views | MAE (UPS): 0.62 -> 0.13 | Helped |
| Replace per-view SVD-based UPS with HO-GSVD (ablation) | MVCPS-NeuS · BUNNY, 4 views | MAE (SDF): 3.00 -> 1.78 | Helped |
| Replace per-view SVD-based UPS with HO-GSVD (ablation) | MVCPS-NeuS · BUNNY, 8 views | MAE (UPS): 0.59 -> 0.085 | Helped |
| Replace per-view SVD-based UPS with HO-GSVD (ablation) | MVCPS-NeuS · BUNNY, 8 views | MAE (SDF): 2.01 -> 1.32 | Helped |
| Replace per-view SVD-based UPS with HO-GSVD (ablation) | MVCPS-NeuS · BUNNY, 20 views | MAE (UPS): 0.421 -> 0.054 | Helped |
Surprising findings
- Plain NeuS with median RGB images fails badly with sparse views, while MonoSDF with monocular priors gives reasonable rough shapes but over-smooths.
Limitations
- Assumes Lambertian reflectance, rarely met in real scenes.
- Non-Lambertian objects under sparse views and lights are left to future work.
- Only a few scenes evaluated; real dataset is small (three objects) and not released.
Experimental setup
Datasets: DiLiGenT-MV BLOBBY BUNNY (Stanford Bunny rendered in Blender) own real-world dataset (RABBIT, CUBE, FROG)
Benchmarks: DiLiGenT-MV BLOBBY (synthetic) BUNNY (synthetic) own real-world dataset
Metrics: Chamfer distance mean angular error
Baselines: NeuS MonoSDF PS-NeRF
Builds on: NeuS PS-NeRF HO-GSVD MonoSDF
Closest papers by meaning
- Uncertainty-Aware Deep Multi-View Photometric Stereo (CVPR 2022 · similarity 0.90)
- RNb-NeuS: Reflectance and Normal-based Multi-View 3D Reconstruction (CVPR 2024 · similarity 0.90)
- MVSAnywhere: Zero-Shot Multi-View Stereo (CVPR 2025 · similarity 0.86)
- Sparse Views Near Light: A Practical Paradigm for Uncalibrated Point-light Photometric Stereo (CVPR 2024 · similarity 0.86)
- NeRSP: Neural 3D Reconstruction for Reflective Objects with Sparse Polarized Images (CVPR 2024 · similarity 0.85)
- SD-MVS: Segmentation-Driven Deformation Multi-View Stereo with Spherical Refinement and EM Optimization (AAAI 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.