Research map / 3D vision / NeRF and radiance fields
NeuRAD: Neural Rendering for Autonomous Driving
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes NeuRAD, a neural feature field with actor-aware 4D hash encoding and explicit sensor modeling (rolling shutter, beam divergence, lidar ray drop) for editable camera and lidar novel view synthesis in driving scenes, evaluated on five AD datasets.
Problem
NeRF-based neural rendering for autonomous driving is slow to train, needs dense semantic supervision, or generalizes poorly across datasets and multi-camera 360° rigs, and lacks realistic lidar modeling.
Approach
A single unified neural feature field with hash-grid features, where static and dynamic actors are separated only by an actor-aware 4D hash encoding; adds a pyramid-based downweighting anti-aliasing scheme, rolling-shutter ray origins/times, and learned lidar intensity and ray-drop heads, plus a CNN upsampler for camera features.
Key results
- On PandaSet front-camera setting NeuRAD reaches 26.58 PSNR vs 25.63 for UniSim; on KITTI MOT it reaches 27.00 PSNR vs 24.00 for MARS.
- Lidar depth median error on PandaSet drops to 0.01 m versus 0.10 m for UniSim, with 96.2% ray drop accuracy.
- Training takes about 1 hour on a single A100, versus about 17 hours for S-NeRF on nuScenes, while matching its nuScenes performance.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace UniSim* with NeuRAD | PandaSet FC novel-view rendering · PandaSet FC | PSNR: 25.44 -> 26.58 | Helped |
| replace UniSim* with NeuRAD | PandaSet 360 novel-view rendering · PandaSet 360 | PSNR: 23.50 -> 25.97 | Helped |
| replace S-NeRF with NeuRAD | nuScenes novel-view rendering · nuScenes | PSNR: 26.21 -> 26.99 | Helped |
| replace MARS with NeuRAD | KITTI MOT novel-view rendering · KITTI MOT | PSNR: 24.00 -> 27.00 | Helped |
| replace UniSim* with NeuRAD | Argoverse 2 novel-view rendering · Argoverse 2 | PSNR: 23.22§ -> 26.22 | Helped |
| replace UniSim* with NeuRAD | ZOD novel-view rendering · ZOD | PSNR: 27.97 -> 29.49 | Helped |
| add CNN decoder (ablation) | NeuRAD · 20-sequence average | PSNR: 25.29 -> 27.26 | Helped |
| add rolling shutter modeling (ablation) | NeuRAD · 20-sequence average | PSNR: 26.77 -> 27.26 | Helped |
| add frustum-based hash-grid downweighting (ablation) | NeuRAD · 20-sequence average | PSNR: 26.12 -> 27.26 | Helped |
| add per-sensor appearance embeddings (ablation) | NeuRAD · 20-sequence average | PSNR: 25.50 -> 27.26 | Helped |
| add supervision from lidar rays with no return (ablation) | NeuRAD · 20-sequence average | PSNR: 25.36 -> 27.26 | Helped |
| replace individual actor hash grids with one shared 4D actor hash grid (ablation) | NeuRAD · 20-sequence average | PSNR: 27.22 -> 27.26 | No clear effect |
Surprising findings
- Ablations show only a modest PSNR impact from rolling shutter modeling, though qualitative renderings and geometry degrade markedly without it.
Limitations
- Assumes actors are rigid and does not support deformations.
- Modeling assumptions are invalid for harsh weather such as heavy rain or snow.
- ZOD sequences rely on pseudo-annotations from an off-the-shelf detector and tracker.
Experimental setup
Datasets: nuScenes PandaSet Argoverse 2 KITTI ZOD
Benchmarks: PandaSet nuScenes KITTI Argoverse 2 ZOD
Metrics: PSNR SSIM LPIPS depth error intensity RMSE ray drop accuracy Chamfer distance FID
Baselines: UniSim S-NeRF Mip-NeRF 360 MARS SUDS Instant-NGP
Builds on: Instant-NGP Zip-NeRF UniSim Neural Scene Graphs Nerfstudio
Closest papers by meaning
- NeRF-LiDAR: Generating Realistic LiDAR Point Clouds with Neural Radiance Fields (AAAI 2024 · similarity 0.89)
- SplatAD: Real-Time Lidar and Camera Rendering with 3D Gaussian Splatting for Autonomous Driving (CVPR 2025 · similarity 0.89)
- LidaRF: Delving into Lidar for Neural Radiance Field on Street Scenes (CVPR 2024 · similarity 0.88)
- Ced-NeRF: A Compact and Efficient Method for Dynamic Neural Radiance Fields (AAAI 2024 · similarity 0.87)
- UC-NERF: Neural Radiance Field for Under-Calibrated Multi-View Cameras in Autonomous Driving (ICLR 2024 · similarity 0.87)
- NeRF On-the-go: Exploiting Uncertainty for Distractor-free NeRFs in the Wild (CVPR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.