Research map / Detection and segmentation / 3D detection and driving
OLiDM: Object-aware LiDAR Diffusion Models for Autonomous Driving
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes OLiDM, an object-aware LiDAR diffusion framework that first generates controllable foreground objects (text and 3D box conditions) and then uses them to condition range-image scene generation, with a semantic-subspace loss (OSA) for alignment.
Problem
Existing LiDAR diffusion models generate whole scenes and produce low-quality, few, and poorly controllable foreground objects, which hurts downstream 3D perception.
Approach
A progressive object-to-scene pipeline: an object diffusion denoiser conditioned on CLIP text embeddings and Fourier-embedded 3D boxes generates LiDAR objects, which a ControlNet-style scene controller injects into a range-image scene denoiser; an Object Semantic Alignment loss applies per-category masks to the diffusion loss to reduce foreground-background misalignment.
Key results
- On KITTI-360 scene generation, OLiDM reaches FPD 7.60 vs 25.1 for UltraLiDAR and JSD 0.42 (best), per the paper's table.
- On foreground object quality, OLiDM achieves CD 1.88 and detects 7.10 boxes per scene vs 9.06 in real KITTI-360 data.
- Sparse-to-dense completion semantic IoU rises to 79.93% vs 34.44% for R2DM; augmenting nuScenes detectors gives +2.4% mAP and +1.9% NDS over baselines.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use OLiDM instead of R2DM | LiDAR scene generation · KITTI-360 | JSD: 0.49 -> 0.42 | Helped |
| Use OLiDM instead of UltraLiDAR | LiDAR scene generation · KITTI-360 | MMD: 1.96 -> 3.45 | Hurt |
| Use OLiDM instead of UltraLiDAR | LiDAR scene generation · KITTI-360 | FPD: 25.1 -> 7.60 | Helped |
| Use OLiDM instead of R2DM | foreground object generation · KITTI-360 foreground objects | CD: 17.6 -> 1.88 | Helped |
| Use OLiDM instead of R2DM | foreground object generation · KITTI-360 foreground objects | #Box: 4.66 -> 7.10 | Helped |
| Use OLiDM instead of R2DM | sparse-to-dense LiDAR completion · KITTI-360 | Semantic IoU (%): 34.44 -> 79.93 | Helped |
| Use OLiDM augmentation instead of LiDAR-Aug | PointPillars · nuScenes | mAP: 45.6 -> 46.7 | Helped |
| Use OLiDM augmentation instead of LiDAR-Aug | CenterPoint · nuScenes | NDS: 67.3 -> 68.5 | Helped |
| Add OSA (ablation) | OLiDM scene generation · KITTI-360 | JSD: 0.47 -> 0.42 | Helped |
| Add text and 3D box conditions (ablation) | OLiDM object generation · KITTI-360 foreground objects | CD: 14.3 -> 1.88 | Helped |
| Add 3D box condition to text condition (ablation) | OLiDM object generation · KITTI-360 foreground objects | CD: 9.21 -> 1.88 | Helped |
| Add text condition to 3D box condition (ablation) | OLiDM object generation · KITTI-360 foreground objects | CD: 4.32 -> 1.88 | Helped |
Surprising findings
- UltraLiDAR achieves the best MMD on KITTI-360 despite poor FPD and object-level quality, showing that distribution metrics disagree with object quality.
Limitations
- Inference is about 16% slower than R2DM.
- Object-level evaluation relies on a detector trained on KITTI, so detected-box counts depend on that detector.
- Object denoiser built on DiT-3D with 12 layers; scale of gains on other datasets is not fully explored.
Experimental setup
Datasets: KITTI-360 nuScenes KITTI
Benchmarks: KITTI-360 nuScenes
Metrics: JSD MMD FPD Chamfer Distance Semantic Similarity #Box MSE IoU mAP NDS
Baselines: LiDARGen UltraLiDAR R2DM LiDAR-Aug GT-Aug LiDARVAE Proj.GAN
Builds on: R2DM LiDARGen DiT-3D ControlNet
Closest papers by meaning
- UniScene: Unified Occupancy-centric Driving Scene Generation (CVPR 2025 · similarity 0.88)
- Training an Open-Vocabulary Monocular 3D Detection Model without 3D Data (NeurIPS 2024 · similarity 0.88)
- An Empirical Study of the Generalization Ability of Lidar 3D Object Detectors to Unseen Domains (CVPR 2024 · similarity 0.88)
- OpenAD: Open-World Autonomous Driving Benchmark for 3D Object Detection (NeurIPS 2025 · similarity 0.87)
- UnO: Unsupervised Occupancy Fields for Perception and Forecasting (CVPR 2024 · similarity 0.87)
- Is Your LiDAR Placement Optimized for 3D Scene Understanding? (NeurIPS 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.