Research map / Detection and segmentation / 3D detection and driving
Uni$^2$Det: Unified and Universal Framework for Prompt-Guided Multi-dataset 3D Detection
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Uni$^2$Det, a unified and universal multi-dataset 3D object detection framework that adds dataset-attribute prompts at voxelization (mean-shifted batch norm), backbone (BEV range masks), and head (object-conditional residuals) to train one shared detector across KITTI, Waymo and nuScenes and generalize zero-shot.
Problem
Naively merging LiDAR 3D detection datasets with different point distributions, point ranges and taxonomies causes negative transfer, and prior multi-dataset methods use dataset-specific heads/normalization that hinder unified training and generalization to unseen datasets.
Approach
Multi-stage prompting modules built from dataset attributes: mean-shifted batch normalization at voxelization for point distribution correction, concatenated BEV point-range masks in 2D backbone layers, and a stop-gradient object-conditional residual added to RoI features with a dataset discriminator, allowing a fully shared detector and head that plug into PV-RCNN and Voxel-RCNN.
Key results
- On Waymo+nuScenes to KITTI zero-shot, Uni2Det (w/ SN) with PV-RCNN reaches 75.57/64.09 mAP vs 72.43/59.91 for data merging w/ SN.
- Joint training on KITTI-Waymo with Voxel-RCNN: Uni2Det achieves 75.25/71.48 on KITTI and 73.01/70.57 on Waymo (car/vehicle-level mAP), above Uni3D's 74.02/68.57 and 72.15/69.46 respectively.
- Ablation on KITTI-Waymo: voxelization prompt alone raises KITTI from 72.73/69.94 to 73.84/70.56; all three stages reach 75.25/71.48.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use Uni²Det instead of Uni3D for joint training | Voxel-RCNN · Waymo-nuScenes consolidation, Waymo validation | mAP (AP/APH): 71.91/68.88 -> 73.27/70.99 | Helped |
| Use Uni²Det instead of Uni3D for joint training | Voxel-RCNN · Waymo-nuScenes consolidation, nuScenes validation | mAP (AP_BE V/AP_3D): 34.95/26.31 -> 36.20/27.19 | Helped |
| Use Uni²Det instead of Uni3D for joint training | Voxel-RCNN · KITTI-nuScenes consolidation, KITTI validation | mAP (AP_BE V/AP_3D): 74.43/69.83 -> 75.91/71.44 | Helped |
| Use Uni²Det instead of Uni3D for joint training | Voxel-RCNN · KITTI-nuScenes consolidation, nuScenes validation | mAP (AP_BE V/AP_3D): 34.51/25.95 -> 36.58/27.68 | Helped |
| Use Uni²Det instead of Uni3D for joint training | Voxel-RCNN · KITTI-Waymo consolidation, KITTI validation | mAP (AP_BE V/AP_3D): 74.02/68.57 -> 75.25/71.48 | Helped |
| Use Uni²Det instead of Uni3D for joint training | Voxel-RCNN · KITTI-Waymo consolidation, Waymo validation | mAP (AP/APH): 72.15/69.46 -> 73.01/70.57 | Helped |
| Add multi-stage prompts (ablation) | Voxel-RCNN · KITTI-Waymo consolidation, KITTI validation | mAP (AP_BE V/AP_3D): 72.73/69.94 -> 75.25/71.48 | Helped |
| Add multi-stage prompts (ablation) | Voxel-RCNN · KITTI-Waymo consolidation, Waymo validation | mAP (AP/APH): 71.09/69.12 -> 73.01/70.57 | Helped |
| Add voxelization prompt (ablation) | Voxel-RCNN · KITTI-Waymo consolidation, KITTI validation | mAP (AP_BE V/AP_3D): 72.73/69.94 -> 73.84/70.56 | Helped |
| Add BEV-based range masking after the voxelization prompt (ablation) | Voxel-RCNN · KITTI-Waymo consolidation, KITTI validation | mAP (AP_BE V/AP_3D): 73.84/70.56 -> 74.96/70.95 | Helped |
| Add object-conditional residual learning to the detection head (ablation) | Uni²Det · KITTI | object-size distribution match | Helped |
| Use mean-shifted batch normalization instead of baseline normalization (ablation) | Voxel-RCNN · KITTI-Waymo consolidation, KITTI validation | mAP (AP_BE V/AP_3D): 72.73/69.94 -> 73.84/70.56 | Helped |
Surprising findings
- Uni3D's multi-dataset training can underperform single-dataset training on some datasets (e.g., Waymo under Waymo-nuScenes consolidation).
- Uni2Det shows lower AP for Car on nuScenes under KITTI-nuScenes consolidation than Uni3D (58.09/39.68 vs 59.25/41.51).
Limitations
- Relies on an identical set of categories across datasets, so it cannot handle different label spaces with more diverse categories.
- Balancing ratio alpha must be tuned per detector (0.1 for Voxel-RCNN vs 0.5 for PV-RCNN, as stated in the text).
Experimental setup
Datasets: Waymo nuScenes KITTI
Benchmarks: Waymo nuScenes KITTI
Metrics: AP APH AP_BEV AP_3D
Baselines: Uni3D PPT SN Data Merging Source Only Pre-training then fine-tuning (P.T.)
Builds on: Uni3D PPT OpenPCDet PV-RCNN Voxel-RCNN
Closest papers by meaning
- Unified Domain Generalization and Adaptation for Multi-View 3D Object Detection (NeurIPS 2024 · similarity 0.88)
- CP-DETR: Concept Prompt Guide DETR Toward Stronger Universal Object Detection (AAAI 2025 · similarity 0.88)
- UniDet3D: Multi-dataset Indoor 3D Object Detection (AAAI 2025 · similarity 0.87)
- SP3D: Boosting Sparsely-Supervised 3D Object Detection via Accurate Cross-Modal Semantic Prompts (CVPR 2025 · similarity 0.87)
- Align3D-AD: Cross-Modal Feature Alignment and Dual-Prompt Learning for Zero-shot 3D Anomaly Detection (NeurIPS 2026 · similarity 0.86)
- Towards Large-scale 3D Representation Learning with Multi-dataset Point Prompt Training (CVPR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.