Research map / Label-efficient and robust learning / Domain adaptation and generalization
ReSimAD: Zero-Shot 3D Domain Transfer for Autonomous Driving with Source Reconstruction and Target Simulation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes ReSimAD, which reconstructs source-domain (Waymo) scenes as LiDAR-only implicit meshes and renders target-domain-like point clouds in CARLA to train 3D detectors zero-shot for new LiDAR/geography domains.
Problem
Autonomous driving 3D detectors trained on a source domain degrade badly under sensor and geographic domain shifts, and adapting them normally requires costly target-domain data collection and annotation.
Approach
Decouple domain-specific characteristics by reconstructing source-domain scenes into 3D meshes via LiDAR-only implicit neural reconstruction, then re-simulate target-domain LiDAR (sensor configuration, scene layout, traffic) in CARLA from those meshes to train detectors with no real target data.
Key results
- On Waymo-to-KITTI with PV-RCNN++, ReSimAD (zero-shot) reaches 82.06 AP_BEV / 61.65 AP_3D, versus 67.68 / 20.81 for source only and 62.55 / 17.53 for ST3D (UDA).
- On Waymo-to-nuScenes with PV-RCNN++, ReSimAD reaches 40.73 AP_BEV / 23.72 AP_3D, above ST3D UDA (34.68 / 17.17).
- Pre-training on about 100K ReSimAD simulated points gives KITTI AP_BEV/AP_3D overall of 71.74 vs 70.57 from scratch, though ONCE real-data pre-training reaches 73.01.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| train on ReSimAD simulated data instead of evaluating the source-only model | PV-RCNN · Waymo-to-KITTI | AP_3D: 22.01 -> 58.42 | Helped |
| train on ReSimAD simulated data instead of evaluating the source-only model | PV-RCNN++ · Waymo-to-KITTI | AP_3D: 20.81 -> 61.65 | Helped |
| train on ReSimAD simulated data instead of evaluating the source-only model | PV-RCNN · Waymo-to-nuScenes | AP_3D: 17.75 -> 21.33 | Helped |
| train on ReSimAD simulated data instead of evaluating the source-only model | PV-RCNN++ · Waymo-to-nuScenes | AP_3D: 18.77 -> 23.72 | Helped |
| train on ReSimAD simulated data instead of evaluating the source-only model | PV-RCNN and PV-RCNN++ · Waymo-to-ONCE | AP_3D: 39.06 -> 48.59; 34.39 -> 53.91 | Helped |
| use ReSimAD instead of ST3D | PV-RCNN · Waymo-to-KITTI | AP_3D: 64.78 -> 58.42 | Hurt |
| use ReSimAD instead of ST3D | PV-RCNN++ · Waymo-to-KITTI | AP_3D: 17.53 -> 61.65 | Helped |
| use ReSimAD instead of ST3D | PV-RCNN and PV-RCNN++ · Waymo-to-nuScenes | AP_3D: 22.99 -> 21.33; 17.17 -> 23.72 | Mixed |
| use ReSimAD instead of ST3D | PV-RCNN and PV-RCNN++ · Waymo-to-ONCE | AP_3D: 41.53 -> 48.59; 36.79 -> 53.91 | Helped |
| increase simulated pre-training data from 55K to 110K samples | AD-PT · KITTI fine-tuning | Overall AP_BEv / AP_3D: 71.06 -> 71.74 | Helped |
| pre-train on 110K simulated samples instead of training from scratch | AD-PT · KITTI fine-tuning | Overall AP_BEv / AP_3D: 70.57 -> 71.74 | Helped |
Surprising findings
- Zero-shot ReSimAD on Waymo-to-KITTI with PV-RCNN++ exceeds ST3D UDA, which has access to real target data.
- CARLA-default and Sensor-like simulated data sometimes degrade performance relative to source-only (e.g., PV-RCNN++ Waymo-to-KITTI AP_BEV drops to 50.65 from 67.68 for CARLA-default).
Limitations
- Simulated background still differs from real scenes, so zero-shot results remain below oracle fine-tuning (e.g., 61.65 vs 82.50 AP_3D on Waymo-to-KITTI with PV-RCNN).
- Reconstruction quality degrades on some sequences (high RMSE and CD in the appendix table), and dynamic objects are handled by asset substitution.
- Evaluated on a limited set of domain pairs (Waymo source; KITTI, nuScenes, ONCE targets) and a few detectors.
Experimental setup
Datasets: Waymo Open Dataset KITTI nuScenes ONCE
Benchmarks: KITTI nuScenes ONCE Waymo
Metrics: AP_BEV AP_3D closed gap RMSE Chamfer Distance
Baselines: ST3D CARLA-default Sensor-like VDBFusion SECOND-IOU
Builds on: NeuS StreetSurf PCSim CARLA AD-PT
Closest papers by meaning
- NeuRAD: Neural Rendering for Autonomous Driving (CVPR 2024 · similarity 0.86)
- VR-Drive: Viewpoint-Robust End-to-End Driving with Feed-Forward 3D Gaussian Splatting (NeurIPS 2025 · similarity 0.85)
- UniDrive: Towards Universal Driving Perception Across Camera Configurations (ICLR 2025 · similarity 0.85)
- DriveDreamer4D: World Models Are Effective Data Machines for 4D Driving Scene Representation (CVPR 2025 · similarity 0.85)
- DrivingRecon: Large 4D Gaussian Reconstruction Model For Autonomous Driving (NeurIPS 2025 · similarity 0.85)
- ReconDreamer: Crafting World Models for Driving Scene Reconstruction via Online Restoration (CVPR 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.