Research map / 3D vision / Depth and stereo
Blurred LiDAR for Sharper 3D: Robust Handheld 3D Scanning with Diffuse LiDAR and RGB
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes fusing RGB with low-cost diffuse (blurred) SPAD LiDAR histograms in a Gaussian-surfel renderer with a scene-adaptive loss for robust handheld 3D reconstruction in low-texture, low-light, low-albedo scenes.
Problem
RGB-based 3D reconstruction fails in low-texture, low-light, and low-albedo scenes, and sparse handheld LiDAR gives poor scene coverage with limited input views.
Approach
Uses wide-field-of-view diffuse LiDAR flash measurements (spatially blurred time-of-flight histograms) rendered via differentiable surfel histogram rasterization, and a scene-adaptive loss that weights RGB patches by texture and SNR so diffuse LiDAR dominates where RGB is uninformative.
Key results
- On no-texture scenes, the method reduces depth MAE to about 0.041-0.045 versus 0.106-0.125 for surfels with sparse LiDAR (Table 1d).
- LiDAR-only depth MAE on 10 training views: diffuse LiDAR 0.036-0.041 vs sparse-point surfels 0.074-0.099 and TransientNeRF 0.221-0.281 (Table 2).
- In textured-object-on-textureless-plane scenes, the method improves depth MAE over sparse LiDAR (e.g. 0.024 vs 0.080 on Lego).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use RGB with diffuse LiDAR instead of RGB with sparse LiDAR | Gaussian surfel reconstruction · No-texture rendered scenes | D.MAE: 0.125 -> 0.045; 0.107 -> 0.045; 0.111 -> 0.041; 0.106 -> 0.042 | Helped |
| use RGB with diffuse LiDAR instead of RGB with monocular depth | Gaussian surfel reconstruction · Textured-object rendered scenes | D.MAE: 0.129 -> 0.033; 0.452 -> 0.030; 0.065 -> 0.024; 0.052 -> 0.024 | Helped |
| use RGB with diffuse LiDAR instead of RGB with monocular depth | Gaussian surfel reconstruction · Full-texture rendered scenes | D.MAE: 0.033 -> 0.025; 0.028 -> 0.017; 0.031 -> 0.016; 0.032 -> 0.025 | Helped |
| use RGB with diffuse LiDAR instead of RGB with monocular depth | Gaussian surfel reconstruction · Textured-plane rendered scenes | D.MAE: 0.036 -> 0.045; 0.049 -> 0.037; 0.050 -> 0.034; 0.055 -> 0.046 | Mixed |
| use diffuse LiDAR loss instead of sparse point LiDAR loss (ablation) | LiDAR-only surfel reconstruction · Rendered scenes | Depth MAE: 0.078 -> 0.041; 0.099 -> 0.040; 0.084 -> 0.036; 0.074 -> 0.037 | Helped |
| add scene-adaptive loss weighting (ablation) | RGB + diffuse LiDAR reconstruction · Rendered scenes | reconstruction quality | Helped |
Surprising findings
- Diffuse (blurred) LiDAR outperforms conventional sparse spot LiDAR for reconstruction despite spatial blur, in limited-view settings.
- Sparse-histogram TransientNeRF performs worse than point-depth sparse LiDAR surfels in the LiDAR-only ablation.
Limitations
- Assumes static, non-specular scenes and known poses estimated from RGB.
- Fails to separate object and plane in very low light, albedo, and texture conditions (Blundstone example).
- Relies on low-resolution 8x8 diffuse LiDAR zones and a short-range sensor mode.
Experimental setup
Datasets: Blender Ball Chair Hotdog Lego (rendered synthetic scenes) Real captures with AMS TMF8828 diffuse LiDAR and RealSense D435i
Benchmarks: Rendered Blender scenes (Ball, Chair, Hotdog, Lego) with four texture variations Real-world handheld captures
Metrics: PSNR SSIM depth MAE normal MAE
Baselines: Gaussian surfels RGB only Surfels with monocular depth Surfels with sparse LiDAR TransientNeRF
Builds on: 2D Gaussian Surfels 3D Gaussian Splatting TransientNeRF COLMAP
Closest papers by meaning
- LiDAR-RT: Gaussian-based Ray Tracing for Dynamic LiDAR Re-simulation (CVPR 2025 · similarity 0.87)
- PlatoNeRF: 3D Reconstruction in Plato's Cave via Single-View Two-Bounce Lidar (CVPR 2024 · similarity 0.86)
- Scaling Diffusion Models to Real-World 3D LiDAR Scene Completion (CVPR 2024 · similarity 0.86)
- Towards Realistic Scene Generation with LiDAR Diffusion Models (CVPR 2024 · similarity 0.85)
- Snapshot Lidar: Fourier Embedding of Amplitude and Phase for Single-Image Depth Reconstruction (CVPR 2024 · similarity 0.85)
- Flexible Depth Completion for Sparse and Varying Point Densities (CVPR 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.