Research map / 3D vision / Depth and stereo
Towards 3D Vision with Low-Cost Single-Photon Cameras
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Reconstructs 3D shape of Lambertian objects from transient histograms of spatially distributed low-cost SPAD proximity sensors by combining a differentiable transient formation model with neural SDF volume rendering.
Problem
Recovering 3D geometry from low-cost, miniature single-photon (SPAD) proximity sensors whose transient histograms have low spatial and temporal resolution, ambient light, and non-idealities such as pile-up, where conventional depth ranging does not apply.
Approach
Model the full transient formation process (laser impulse, pile-up, jitter, ambient offset) as a differentiable forward model and fit a NeuS-style neural SDF by rendering transients and minimizing the histogram difference from multiple posed sensors, with bilevel importance sampling over rays and bins.
Key results
- On simulated data the method reaches an average Chamfer distance below 5 mm (e.g., Bunny 3.84 mm), an order of magnitude below the best baseline (space carving, ~21-45 mm).
- On real captures with the $10 AMS TMF8820 sensor, two-way Chamfer for the method is 6-18 mm across objects, versus 17-99 mm for baselines (e.g., Toy Container 11.5 vs 51.6 mm best baseline).
- Method remains robust to non-Lambertian objects with rich texture (toy container, cereal box, spray bottle) but fails on highly specular objects.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use the proposed transient-rendering reconstruction method instead of space carving | 3D reconstruction from low-cost SPAD measurements · Armadillo (simulated) | Chamfer distance (mm): 34.78 -> 3.93 | Helped |
| Use the proposed transient-rendering reconstruction method instead of space carving | 3D reconstruction from low-cost SPAD measurements · Bear (simulated) | Chamfer distance (mm): 24.53 -> 5.95 | Helped |
| Use the proposed transient-rendering reconstruction method instead of space carving | 3D reconstruction from low-cost SPAD measurements · Bunny (simulated) | Chamfer distance (mm): 22.29 -> 3.84 | Helped |
| Use the proposed transient-rendering reconstruction method instead of space carving | 3D reconstruction from low-cost SPAD measurements · Digit (simulated) | Chamfer distance (mm): 45.44 -> 3.27 | Helped |
| Use the proposed transient-rendering reconstruction method instead of space carving | 3D reconstruction from low-cost SPAD measurements · Einstein (simulated) | Chamfer distance (mm): 26.60 -> 3.51 | Helped |
| Use the proposed transient-rendering reconstruction method instead of space carving | 3D reconstruction from low-cost SPAD measurements · Skull (simulated) | Chamfer distance (mm): 25.49 -> 3.22 | Helped |
| Use the proposed transient-rendering reconstruction method instead of space carving | 3D reconstruction from low-cost SPAD measurements · Soap (simulated) | Chamfer distance (mm): 21.44 -> 3.23 | Helped |
| Use the proposed transient-rendering reconstruction method instead of space carving | 3D reconstruction from low-cost SPAD measurements · Sphere (simulated) | Chamfer distance (mm): 25.47 -> 3.77 | Helped |
| Use the proposed transient-rendering reconstruction method instead of reprojection with thresholding | 3D reconstruction from low-cost SPAD measurements · Big Box (real-world) | Chamfer distance (mm): 67.5 -> 12.5 | Helped |
| Use the proposed transient-rendering reconstruction method instead of reprojection with thresholding | 3D reconstruction from low-cost SPAD measurements · Block (real-world) | Chamfer distance (mm): 52.3 -> 9.8 | Helped |
| Use the proposed transient-rendering reconstruction method instead of reprojection with thresholding | 3D reconstruction from low-cost SPAD measurements · Pyramid (real-world) | Chamfer distance (mm): 75.4 -> 18.4 | Helped |
| Use the proposed transient-rendering reconstruction method instead of reprojection with thresholding | 3D reconstruction from low-cost SPAD measurements · Toy Container (real-world) | Chamfer distance (mm): 52.4 -> 11.5 | Helped |
Surprising findings
- Method is robust to strong ambient light and mildly specular surfaces despite assuming a Lambertian BRDF with uniform albedo.
Limitations
- Fails on highly specular objects, as a Lambertian model is assumed.
- Requires hundreds of views for complex shapes and takes hours per reconstruction.
- Limited sensor range (4-6 m) and temporal resolution, so unproven for larger scenes or finer details.
Experimental setup
Datasets: simulated transients for 8 scenes (Armadillo, Bear, Bunny, Digit, Einstein, Skull, Soap, Sphere) real-world captures of 8 tabletop objects using TMF8820 on UR5 arm
Benchmarks: simulated transient dataset (8 objects) real-world tabletop captures (8 objects)
Metrics: Chamfer distance (mm) one-way Chamfer (real-world)
Baselines: Reprojection (peak) Reprojection (threshold) Space carving
Builds on: NeuS NeRF Transient NeRF Jungerman et al. planar SPAD reconstruction Sifferman et al. multi-pixel SPAD planar reconstruction
Closest papers by meaning
- PlatoNeRF: 3D Reconstruction in Plato's Cave via Single-View Two-Bounce Lidar (CVPR 2024 · similarity 0.86)
- Opportunistic Single-Photon Time of Flight (CVPR 2025 · similarity 0.85)
- TurboSL: Dense Accurate and Fast 3D by Neural Inverse Structured Light (CVPR 2024 · similarity 0.84)
- Real-time Acquisition and Reconstruction of Dynamic Volumes with Neural Structured Illumination (CVPR 2024 · similarity 0.84)
- Generalized Event Cameras (CVPR 2024 · similarity 0.84)
- Sparse Views Near Light: A Practical Paradigm for Uncalibrated Point-light Photometric Stereo (CVPR 2024 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.