Research map / 3D vision / 3D generation
UniSDF: Unifying Neural Representations for High-Fidelity 3D Reconstruction of Complex Scenes with Reflections
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes UniSDF, which learns a 3D weight field to blend camera-view and reflected-view radiance fields on an SDF-based iNGP hash-grid backbone, giving accurate surfaces for scenes with both reflective and non-reflective regions.
Problem
Neural implicit surface reconstruction of large complex real-world scenes with reflections: camera-view radiance fields are robust but miss reflections, while reflected-view radiance fields capture specular reflections but are unstable and oversmooth on real scenes.
Approach
Combine a camera-view and a reflected-view radiance field via a learned, unsupervised 3D weight field that composes their colors per pixel after volume rendering, on a shared SDF hash-grid backbone trained coarse-to-fine, so each surface region uses the better parameterization.
Key results
- Achieves Chamfer distance 0.64 mm on DTU, second best behind Geo-NeuS (0.51 mm) and better than other methods listed.
- Averaged across datasets, reaches PSNR 29.40, SSIM 0.807, LPIPS 0.174, the best mean among compared methods.
- Outperforms the single-field custom baselines CamV and RefV on DTU, Mip-NeRF 360 and Ref-NeRF real in all metrics.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Combine camera-view and reflected-view radiance fields with a learned weight field instead of using only one field (ablation) | UniSDF · DTU | Chamfer Distance (mm): 0.85 -> 0.64 | Helped |
| Combine camera-view and reflected-view radiance fields with a learned weight field instead of using only one field (ablation) | UniSDF · Mip-NeRF 360 | PSNR: 27.26 -> 27.67 | Helped |
| Combine camera-view and reflected-view radiance fields with a learned weight field instead of using only one field (ablation) | UniSDF · Ref-NeRF real | PSNR: 23.30 -> 23.70 | Helped |
| Use coarse-to-fine hash-grid training rather than activating all feature pyramid grids at the beginning (ablation) | UniSDF · Ref-NeRF real, sedan | reconstruction quality | Helped |
| Use separate camera-view and reflected-view radiance fields with learnable weights rather than one field receiving both directions (ablation) | UniSDF · Ref-NeRF real | reconstruction quality | Helped |
| Use the SDF-gradient normal instead of the predicted normal to compute reflected view direction and the loss (ablation) | UniSDF · Rendering evaluation | rendering performance | Helped |
Surprising findings
- Geo-NeuS, the best method on DTU, performs worse on Shiny Blender, and NeRO and Ref-NeuS underperform their NeuS baseline on general real scenes.
- Separate diffuse and specular components in the reflected-view branch can cause optimization failures where the specular component stays blank and geometry is wrong.
Limitations
- Ref-NeRF real results are weaker than Zip-NeRF in PSNR, and surface quality on some scenes is limited.
- BakedSDF baseline required per-scene eikonal weight tuning, indicating sensitivity of baselines; UniSDF itself is not shown to be free of such tuning.
Experimental setup
Datasets: DTU Shiny Blender Mip-NeRF 360 Ref-NeRF real
Benchmarks: DTU Shiny Blender Mip-NeRF 360 Ref-NeRF real
Metrics: Chamfer distance PSNR SSIM LPIPS MAE accuracy
Baselines: Mip-NeRF 360 Zip-NeRF Neuralangelo Geo-NeuS Ref-NeRF ENVIDR NeRO Ref-NeuS Factored-NeuS BakedSDF NeuS NeuralWarp
Builds on: NeRF VolSDF Instant-NGP Ref-NeRF Mip-NeRF 360
Closest papers by meaning
- AniSDF: Fused-Granularity Neural Surfaces with Anisotropic Encoding for High-Fidelity 3D Reconstruction (ICLR 2025 · similarity 0.89)
- GNeRP: Gaussian-guided Neural Reconstruction of Reflective Objects with Noisy Polarization Priors (ICLR 2024 · similarity 0.89)
- Decompositional Neural Scene Reconstruction with Generative Diffusion Prior (CVPR 2025 · similarity 0.89)
- NeRSP: Neural 3D Reconstruction for Reflective Objects with Sparse Polarized Images (CVPR 2024 · similarity 0.88)
- 3D Reconstruction with Generalizable Neural Fields using Scene Priors (ICLR 2024 · similarity 0.88)
- Differentiable Neural Surface Refinement for Modeling Transparent Objects (CVPR 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.