Research map / 3D vision / NeRF and radiance fields
LiDAR-RT: Gaussian-based Ray Tracing for Dynamic LiDAR Re-simulation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes LiDAR-RT, a Gaussian-primitive LiDAR representation rendered via hardware-accelerated OptiX ray tracing with scene graphs, enabling real-time dynamic LiDAR re-simulation.
Problem
Real-time, physically accurate LiDAR re-simulation in dynamic driving scenes; prior NeRF-based LiDAR methods are too slow for training and rendering in large-scale scenes.
Approach
Represent LiDAR properties (intensity, ray-drop) as learnable SH coefficients on 3D Gaussians split into background and object models, and render range images by ray tracing Gaussian proxy geometries (co-planar triangle pairs in a BVH) with a chunked sorted-buffer OptiX pipeline and front-to-back differentiable backward pass.
Key results
- On Waymo, LiDAR-RT reaches 20.1 FPS versus 0.17 for LiDAR4D and depth PSNR 22.158 vs 21.841, with 1.37 GB storage vs 7.7 GB.
- On KITTI-360, LiDAR-RT reaches 42.7 FPS versus 0.4 for LiDAR4D, with depth PSNR 27.676 vs 27.477 and CD 0.1077 vs 0.1089.
- Ablations on KITTI-360 show removing normal-guided initialization or the spatial ray-drop module degrades geometry and rendering quality.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use LiDAR-RT instead of LiDAR4D | LiDAR re-simulation · Waymo Open | Depth RMSE: 6.6234 -> 6.4577 | Helped |
| Use LiDAR-RT instead of LiDAR4D | LiDAR re-simulation · KITTI-360 | Depth RMSE: 3.5256 -> 3.4671 | Helped |
| Add Chamfer Distance loss (ablation) | LiDAR-RT · KITTI-360 dynamic sequence | Depth RMSE: 3.5053 -> 3.4671 | Helped |
| Add normal-guided point-cloud initialization (ablation) | LiDAR-RT · KITTI-360 dynamic sequence | Depth RMSE: 4.0597 -> 3.4671 | Helped |
| Add spatial perception ray-drop modeling instead of naive ray-drop estimation (ablation) | LiDAR-RT · KITTI-360 dynamic sequence | Depth RMSE: 3.7445 -> 3.4671 | Helped |
Surprising findings
- On Waymo, intensity PSNR (23.91) is slightly below LiDAR4D (24.08) despite better depth metrics.
Limitations
- Cannot accurately model non-rigid objects such as pedestrians and cyclists.
- Performance and rendering speed degrade on long driving sequences due to growing Gaussian count.
- Lacks material estimation and reflection modeling for more physically accurate LiDAR.
Experimental setup
Datasets: Waymo Open Dataset KITTI-360
Benchmarks: Waymo Open Dataset KITTI-360
Metrics: RMSE MedAE LPIPS SSIM PSNR Chamfer Distance F-score FPS storage
Baselines: LiDARsim PCGen LiDAR-NeRF LiDAR4D DyNFL
Builds on: 3D Gaussian Splatting 3DGRT 2D Gaussian Splatting LiDAR4D LiDAR-NeRF
Closest papers by meaning
- SplatAD: Real-Time Lidar and Camera Rendering with 3D Gaussian Splatting for Autonomous Driving (CVPR 2025 · similarity 0.90)
- GS-LiDAR: Generating Realistic LiDAR Point Clouds with Panoramic Gaussian Splatting (ICLR 2025 · similarity 0.89)
- Dynamic LiDAR Re-simulation using Compositional Neural Fields (CVPR 2024 · similarity 0.89)
- STGC-NeRF: Spatial-Temporal Geometric Consistency for LiDAR Neural Radiance Fields in Dynamic Scenes (AAAI 2025 · similarity 0.88)
- LidaRF: Delving into Lidar for Neural Radiance Field on Street Scenes (CVPR 2024 · similarity 0.88)
- Blurred LiDAR for Sharper 3D: Robust Handheld 3D Scanning with Diffuse LiDAR and RGB (CVPR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.