Research map / 3D vision / Gaussian splatting
SplatAD: Real-Time Lidar and Camera Rendering with 3D Gaussian Splatting for Autonomous Driving
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SplatAD, the first 3D Gaussian Splatting method that renders both camera and lidar data for dynamic driving scenes in real time, with sensor-specific rolling shutter, intensity, and ray-drop modeling.
Problem
Sensor-realistic camera and lidar simulation for autonomous driving is slow with NeRFs, while existing 3DGS methods render only camera data and cannot handle lidar.
Approach
A unified 3D Gaussian representation with per-Gaussian features replacing spherical harmonics, custom CUDA rasterizers for lidar in spherical coordinates with sensor-specific tiling, and rolling-shutter compensation applied in 2D/sensor space for both camera and lidar.
Key results
- On PandaSet NVS, SplatAD reaches 26.70 PSNR and 104.1 MP/s vs NeuRAD's 25.80 PSNR and 9.7 MP/s.
- Lidar NVS on nuScenes: SplatAD achieves 0.02 depth error and 5.7 MR/s, comparable to NeuRAD's accuracy while running about 5x faster in rendering.
- Reconstruction on PandaSet: SplatAD reaches 29.74 PSNR, the best among compared methods, and trains in about 1 hour on a single A100.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use SplatAD instead of NeuRAD | SplatAD · PandaSet NVS images | PSNR: 25.80 -> 26.70 | Helped |
| use SplatAD instead of NeuRAD | SplatAD · Argo2 NVS images | PSNR: 26.18 -> 28.40 | Helped |
| use SplatAD instead of NeuRAD | SplatAD · nuScenes NVS images | PSNR: 26.17 -> 27.53 | Helped |
| add camera rolling shutter modeling (ablation) | SplatAD · PandaSet, nuScenes, Argoverse2 NVS | PSNR: 27.48 -> 27.54 | Helped |
| use a CNN instead of an MLP for image decoding (ablation) | SplatAD · PandaSet, nuScenes, Argoverse2 NVS | PSNR: 27.23 -> 27.54 | Helped |
| add EWA antialiasing (ablation) | SplatAD · PandaSet, nuScenes, Argoverse2 NVS | LPIPS: 0.268 -> 0.255 | Helped |
| add sensor appearance embeddings (ablation) | SplatAD · PandaSet, nuScenes, Argoverse2 NVS | PSNR: 27.28 -> 27.54 | Helped |
| add velocity optimization (ablation) | SplatAD · PandaSet, nuScenes, Argoverse2 NVS | PSNR: 27.55 -> 27.54 | No clear effect |
| add lidar rolling shutter compensation (ablation) | SplatAD · PandaSet, nuScenes, Argoverse2 NVS | PSNR: 27.42 -> 27.54 | Helped |
| use MCMC (ablation) | SplatAD · PandaSet, nuScenes, Argoverse2 NVS | PSNR: 27.35 -> 27.54 | Helped |
| use median depth instead of expected depth at inference (ablation) | SplatAD · PandaSet, nuScenes, Argoverse2 lidar NVS | Chamfer distance: 4.9 -> 2.0 | Helped |
Surprising findings
- Removing rolling shutter compensation barely changes rendering speed, indicating the approximation is efficient.
- Median depth rather than expected depth at inference substantially improves lidar Chamfer distance (2.0 vs 4.9 in ablation).
Limitations
- Dynamic actors are modeled as rigid bodies only.
- Extrapolation to shifted views is weaker than the lidar/camera NVS results suggest; diffusion priors are proposed as a remedy.
- Baselines lack lidar intensity and ray-drop outputs, so comparisons on those metrics are incomplete.
Experimental setup
Datasets: PandaSet Argoverse2 nuScenes
Benchmarks: PandaSet Argoverse2 nuScenes
Metrics: PSNR SSIM LPIPS depth error intensity error ray drop accuracy chamfer distance FD DINOv2 MP/s MR/s
Baselines: UniSim NeuRAD PVG Street Gaussians OmniRe
Builds on: 3D Gaussian Splatting NeuRAD gsplat 3DGS-MCMC Street Gaussians
Closest papers by meaning
- DrivingGaussian: Composite Gaussian Splatting for Surrounding Dynamic Autonomous Driving Scenes (CVPR 2024 · similarity 0.90)
- LiDAR-RT: Gaussian-based Ray Tracing for Dynamic LiDAR Re-simulation (CVPR 2025 · similarity 0.90)
- SplatFormer: Point Transformer for Robust 3D Gaussian Splatting (ICLR 2025 · similarity 0.89)
- GS-LiDAR: Generating Realistic LiDAR Point Clouds with Panoramic Gaussian Splatting (ICLR 2025 · similarity 0.89)
- FlexDrive: Toward Trajectory Flexibility in Driving Scene Gaussian Splatting Reconstruction and Rendering (CVPR 2025 · similarity 0.89)
- NeuRAD: Neural Rendering for Autonomous Driving (CVPR 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.