Research map / 3D vision / NeRF and radiance fields
PlatoNeRF: 3D Reconstruction in Plato's Cave via Single-View Two-Bounce Lidar
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes PlatoNeRF, which trains a NeRF to reproduce single-photon lidar two-bounce transients so that visible and occluded geometry can be reconstructed from a single view without data priors.
Problem
Single-view 3D reconstruction with NeRF is ill-posed; data-prior methods hallucinate occluded geometry, and RGB shadow-based methods fail under ambient light or low albedo.
Approach
Model two-bounce optical paths inside NeRF using primary rays (to estimate the visible depth d3 and two-bounce time of flight) and secondary rays toward a virtual light (to estimate shadows), supervised by single-photon lidar transients, giving physically grounded occluded geometry from one view.
Key results
- PlatoNeRF achieves test-view L1 depth of 0.0862 m on the chair scene versus 0.2151 m for BF Lidar and 0.1336 m for S3-NeRF on the occlusion scene (train/test depth tables reported per scene).
- Chamfer distance of 0.0280 for PlatoNeRF versus 0.0465 for BF Lidar and 0.4129 for S3-NeRF averaged over four simulated scenes.
- At 32x32 spatial resolution PlatoNeRF L1 depth is 0.1070 m versus 0.1799 m for BF Lidar; under ambient light intensity 4 S3-NeRF error rises to 0.3080 m while PlatoNeRF stays at 0.0794 m.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use PlatoNeRF instead of BF Lidar or S3-NeRF for single-view reconstruction | Chair, dragon, bunny, and occlusion scenes · 120 test views of four simulated scenes | L1 depth (m): Chair: 0.1178, 0.1837 -> 0.0862; Dragon: 0.1042, 0.1049 -> 0.0870; Bunny: 0.0877, 0.0660 -> 0.0601; Occlusion: 0.1336, 0 | Helped |
| Use PlatoNeRF instead of BF Lidar or S3-NeRF | Four simulated scenes · All four simulated scenes | Chamfer distance: 0.0465, 0.4129 -> 0.0280 | Helped |
| Reduce sensor spatial resolution to 128 × 128 (ablation) | PlatoNeRF, chair scene · Chair scene, spatial resolution 128 × 128 | L1 depth (m): 0.0880 | Hurt |
| Reduce sensor spatial resolution to 64 × 64 and 32 × 32 (ablation) | PlatoNeRF, chair scene · Chair scene, downsampled spatial resolutions | L1 depth (m): 64 × 64: 0.0932; 32 × 32: 0.1070 | Hurt |
| Increase transient bin size (ablation) | PlatoNeRF, chair scene · Chair scene, temporal resolutions 256 ps, 512 ps, and 1024 ps | L1 depth (m): 256 ps: 0.0965; 512 ps: 0.1210; 1024 ps: 0.1833 | Hurt |
| Add ambient light at intensity 4 (ablation) | PlatoNeRF and S3-NeRF, chair scene · Chair scene, ambient light intensity 4 | L1 depth (m): PlatoNeRF: 0.0862 -> 0.0794; S3-NeRF: 0.1178 -> 0.3080 | Mixed |
| Reduce background scene albedo by 4× (ablation) | PlatoNeRF and S3-NeRF, chair scene · Chair scene, background albedo 4× less | L1 depth (m): PlatoNeRF: 0.0862 -> 0.0859; S3-NeRF: 0.1178 -> 0.2152 | Mixed |
| Reduce illumination spots from 16 to 8 (ablation) | PlatoNeRF · Chair scene, 8 illumination spots | L1 depth (m): 9.12 cm | Hurt |
| Vary the shadow mask threshold between zero and one (ablation) | PlatoNeRF · Chair scene | L1 depth (m): 6.88 and 70 cm | Mixed |
| Use PlatoNeRF on curved, non-planar background walls (ablation) | PlatoNeRF · Non-planar scene, test views | L1 depth (m): 7.75 cm | No clear effect |
Surprising findings
- S3-NeRF does not converge when trained on multi-bounce renderings, so it was trained with one-bounce RGB images only.
- Aligning S3-NeRF's relative-depth output with ICP did not improve its metrics.
Limitations
- Only models Lambertian reflectance; specular surfaces are out of scope.
- Built on vanilla NeRF, which produces occasional floaters.
- Focuses on indoor scenes with laser scanning rather than flash illumination.
Experimental setup
Datasets: simulated single-photon lidar scenes (Mitsuba ToF extension) Henley et al. real single-photon lidar dataset
Benchmarks: PlatoNeRF simulated scenes (chair, dragon, bunny, occlusion) Henley et al. real lidar dataset
Metrics: L1 depth error PSNR Chamfer distance
Baselines: Bounce-Flash Lidar S3-NeRF
Builds on: NeRF Bounce-Flash Lidar S3-NeRF
Closest papers by meaning
- Neural Inverse Rendering from Propagating Light (CVPR 2025 · similarity 0.87)
- LiDAR4D: Dynamic Neural Fields for Novel Space-time View LiDAR Synthesis (CVPR 2024 · similarity 0.87)
- Blurred LiDAR for Sharper 3D: Robust Handheld 3D Scanning with Diffuse LiDAR and RGB (CVPR 2025 · similarity 0.86)
- STGC-NeRF: Spatial-Temporal Geometric Consistency for LiDAR Neural Radiance Fields in Dynamic Scenes (AAAI 2025 · similarity 0.86)
- Gated Fields: Learning Scene Reconstruction from Gated Videos (CVPR 2024 · similarity 0.86)
- Towards 3D Vision with Low-Cost Single-Photon Cameras (CVPR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.