Research map / 3D vision / NeRF and radiance fields
UC-NERF: Neural Radiance Field for Under-Calibrated Multi-View Cameras in Autonomous Driving
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes UC-NeRF, a Zip-NeRF-based method for novel view synthesis from under-calibrated multi-camera driving rigs, combining layer-based color correction, virtual warping, and spatiotemporally constrained pose refinement.
Problem
NeRF training on multi-camera driving data degrades because of per-camera ISP color inconsistency and pose errors from vehicle misalignment and vibration.
Approach
Split scenes into foreground and sky layers with separate affine color corrections decoded from per-image latent codes, augment training with warped virtual views that carry color correction codes, and refine poses by explicit cross-camera and cross-time reprojection constraints tied to a fixed camera-to-ego transform.
Key results
- On Waymo, UC-NeRF reaches 28.13 PSNR / 0.842 SSIM / 0.356 LPIPS versus 26.21 PSNR for Zip-NeRF, the strongest baseline.
- On NuScenes, UC-NeRF reaches 30.20 PSNR / 0.876 SSIM / 0.374 LPIPS versus 27.06 PSNR for Zip-NeRF.
- Ablation on Waymo: layer-based color correction, pose refinement, and virtual warping together raise PSNR from 26.21 to 28.13.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace Zip-NeRF with UC-NeRF | novel view synthesis · Waymo | PSNR: 26.21 -> 28.13 | Helped |
| replace Zip-NeRF with UC-NeRF | novel view synthesis · NuScenes | PSNR: 27.06 -> 30.20 | Helped |
| add layer-based color correction (ablation) | UC-NeRF on Waymo · Waymo | PSNR: 26.21 -> 27.18 | Helped |
| add spatiotemporally constrained pose refinement (ablation) | UC-NeRF with layer-based color correction on Waymo · Waymo | PSNR: 27.18 -> 27.82 | Helped |
| add virtual warping (ablation) | UC-NeRF with layer-based color correction and pose refinement on Waymo · Waymo | PSNR: 27.82 -> 28.13 | Helped |
| add virtual warping without pose refinement (ablation) | UC-NeRF with layer-based color correction on Waymo · Waymo | PSNR: 27.18 -> 27.26 | Helped |
| set sky loss weight to 0.001 instead of 0 (ablation) | UC-NeRF · Waymo | PSNR: 27.89 -> 28.05 | Helped |
| increase sky loss weight from 0.001 to 0.002 (ablation) | UC-NeRF · Waymo | PSNR: 28.05 -> 28.15 | Helped |
| increase sky loss weight from 0.002 to 0.004 (ablation) | UC-NeRF · Waymo | PSNR: 28.15 -> 27.98 | Hurt |
| add spatiotemporal constraint to NeRF-- pose refinement (ablation) | NeRF-- · Waymo | PSNR: 28.48 -> 28.51 | No clear effect |
| add spatiotemporal constraint to explicit-correspondence pose refinement (ablation) | UC-NeRF without STC · Waymo | PSNR: 29.01 -> 29.14 | Helped |
| replace NeRF-- pose refinement with explicit pixel correspondences and spatiotemporal constraint (ablation) | baseline pose refinement · Waymo | PSNR: 28.48 -> 29.14 | Helped |
Surprising findings
- Joint NeRF photometric pose refinement (NeRF--) gives only a small gain over the baseline, while explicit correspondence-based refinement with the spatiotemporal constraint gives a much larger gain.
Limitations
- Evaluated only on static scenes from two driving datasets.
- Virtual view quality depends on pose accuracy and MVS depth quality.
- Training cost (about 3 h per scene) and inference latency are higher than Zip-NeRF and Instant-NGP.
Experimental setup
Datasets: Waymo NuScenes
Benchmarks: Waymo NuScenes
Metrics: PSNR SSIM LPIPS Abs Rel RMSE delta1
Baselines: Mip-NeRF Mip-NeRF 360 Instant-NGP S-NeRF Zip-NeRF NeRF in the wild Urban-NeRF NeRF--
Builds on: Zip-NeRF NeRF in the wild Urban-NeRF NeRF-- S-NeRF
Closest papers by meaning
- Joint Optimization of Neural Radiance Fields and Continuous Camera Motion from a Monocular Video (CVPR 2025 · similarity 0.89)
- Few-Shot Neural Radiance Fields under Unconstrained Illumination (AAAI 2024 · similarity 0.89)
- CF-NeRF: Camera Parameter Free Neural Radiance Fields with Incremental Learning (AAAI 2024 · similarity 0.88)
- NeRF-VPT: Learning Novel View Representations with Neural Radiance Fields via View Prompt Tuning (AAAI 2024 · similarity 0.88)
- USB-NeRF: Unrolling Shutter Bundle Adjusted Neural Radiance Fields (ICLR 2024 · similarity 0.88)
- SOAC: Spatio-Temporal Overlap-Aware Multi-Sensor Calibration using Neural Radiance Fields (CVPR 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.