Research map / Label-efficient and robust learning / Domain adaptation and generalization
LiT: Unifying LiDAR "Languages" with LiDAR Translator
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes LiDAR Translator (LiT), a data-driven framework that reconstructs source scenes (foreground via SDF fitting, background via neural kernel fields) and re-simulates them with target-LiDAR ray angles and learned ray-drop, enabling zero-shot cross-domain detection and multi-source joint training.
Problem
LiDAR perception models suffer large domain gaps across sensors, vehicles, and environments (beam count, FoV, ray-drop, object sizes), which hinder zero-shot transfer and joint training across LiDAR datasets.
Approach
Rather than adapting the perception model, translate source LiDAR scenes into the target LiDAR's characteristics by reconstructing scenes and foreground vehicles, then ray-casting with statistically modeled vertical ray angles and a learned ray-drop predictor on a hardware-accelerated engine.
Key results
- On Waymo→KITTI with SECOND-IoU, LiT reaches 82.55 AP_BEV and 69.94 AP_3D versus 67.64/27.48 without translation and 78.96/59.20 for SN.
- On nuScenes→KITTI with SECOND-IoU, LiT closes 91.26% (AP_BEV) and 76.01% (AP_3D) of the gap to oracle, outperforming ST3D's 76.63%/59.50%.
- Training on Waymo+nuScenes translated to KITTI yields 84.45 AP_BEV with SECOND-IoU, exceeding the oracle's 83.29 without seeing target data.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Translate source LiDAR into target domain with LiT (zero-shot) | SECOND-IoU detector · Waymo→KITTI | AP_BEV car IoU 0.7: 67.64 to 82.55 | Helped |
| Translate source LiDAR into target domain with LiT (zero-shot) | SECOND-IoU detector · nuScenes→KITTI | AP_3D car IoU 0.7: 17.92 to 60.13 | Helped |
| Translate source LiDAR into target domain with LiT (zero-shot) | SECOND-IoU detector · Waymo→nuScenes | AP_BEV car IoU 0.7: 32.91 to 37.00 | Helped |
| Use multi-source LiDAR translation (Waymo+nuScenes) for training | SECOND-IoU detector · W+N→K zero-shot | AP_BEV car IoU 0.7: 84.45 vs oracle 83.29 | Helped |
| Foreground modeling: use 1 shared foreground mesh instead of diverse per-vehicle meshes (ablation) | LiT foreground modeling · nuScenes→KITTI | AP_3D car IoU 0.7: 60.13 to 55.30 | Hurt |
| Foreground-only simulation (no background) (ablation) | LiT scene modeling · nuScenes→KITTI | AP_3D car IoU 0.7: 60.13 to 48.45 | Hurt |
| Add Gaussian noise std 0.05m to foreground mesh vertices (ablation) | LiT foreground modeling · nuScenes→KITTI | AP_3D car IoU 0.7: 60.13 to 35.37 | Hurt |
| Add Gaussian noise std 0.05m to background mesh vertices (ablation) | LiT background modeling · nuScenes→KITTI | AP_3D car IoU 0.7: 60.13 to 58.21 | Hurt |
| Statistical modeling of LiDAR vertical ray angles and ray-drop modeling (ablation) | LiDAR modeling · nuScenes self-to-self translation | Chamfer distance | Helped |
| Statistical ray-angle modeling plus ray-drop in translation vs source baseline | LiT LiDAR modeling · Waymo→nuScenes statistical alignment | MMD_BEV and JSD_BEV: MMD 2.310e-03 to 6.583e-04; JSD 0.380 to 0.205 | Helped |
| Translate LiDAR with LiT vs source baseline for Waymo→KITTI with PV-RCNN | PV-RCNN · Waymo→KITTI | AP_BEV car IoU 0.7: 61.18 to 84.35 | Helped |
Surprising findings
- Multi-source LiT translation surpasses the fully supervised oracle on KITTI with SECOND-IoU in AP_BEV despite zero target-domain training.
- Naively combining Waymo and nuScenes sources without translation does not improve, and can harm, zero-shot performance.
Limitations
- Relies on exclusive LiDAR data and depends on annotated source datasets for foreground reconstruction (tracking/boxes).
- Only vehicle objects are modeled in the foreground; other object types are not handled.
- Detection model is more sensitive to foreground reconstruction inaccuracies than background ones.
Experimental setup
Datasets: KITTI Waymo nuScenes ShapeNetV2
Benchmarks: KITTI Waymo nuScenes
Metrics: AP_BEV AP_3D MMD JSD Chamfer distance
Baselines: Source only SN ST3D ReSimAD Oracle
Builds on: ReSimAD neural kernel field PCGen ST3D SECOND-IoU
Closest papers by meaning
- UniMix: Towards Domain Adaptive and Generalizable LiDAR Semantic Segmentation in Adverse Weather (CVPR 2024 · similarity 0.87)
- Towards foundational LiDAR world models with efficient latent flow matching (NeurIPS 2025 · similarity 0.86)
- LiDAR-RT: Gaussian-based Ray Tracing for Dynamic LiDAR Re-simulation (CVPR 2025 · similarity 0.86)
- LiDAR-LLM: Exploring the Potential of Large Language Models for 3D LiDAR Understanding (AAAI 2025 · similarity 0.86)
- OLiDM: Object-aware LiDAR Diffusion Models for Autonomous Driving (AAAI 2025 · similarity 0.86)
- An Empirical Study of the Generalization Ability of Lidar 3D Object Detectors to Unseen Domains (CVPR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.