Research map / Detection and segmentation / 3D detection and driving
UniDet3D: Multi-dataset Indoor 3D Object Detection
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes UniDet3D, a transformer-encoder-only 3D object detector trained jointly on six indoor datasets with a unified label space, reaching state-of-the-art on all six benchmarks.
Problem
Indoor 3D object detection datasets are individually small and label-inconsistent, so models trained on one dataset generalize poorly; multi-dataset training for indoor 3D detection is underexplored.
Approach
A vanilla self-attention transformer encoder (no positional encoding, no cross-attention decoder) over superpoint-pooled sparse U-Net features, with a unified de-duplicated label space across datasets and disentangled superpoint-based matching replacing Hungarian matching.
Key results
- Jointly trained UniDet3D reaches 77.5 mAP25 / 66.1 mAP50 on ScanNet (best run), exceeding V-DETR's 77.4 / 65.0.
- On S3DIS, UniDet3D reaches 75.2 / 60.8 mAP25/mAP50 (best), +9.1 mAP50 over the second-best reported method.
- Unified label space outperforms partitioned on average across 25 trials (e.g., S3DIS 57.9 vs 56.2 mAP50), while using 99 classes instead of 159.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| jointly train across datasets rather than train separately (ablation) | UniDet3D · S3DIS | mAP50: 39.7 -> 57.9 | Helped |
| use unified instead of partitioned label space (ablation) | UniDet3D joint training · S3DIS | mAP50: 56.2 -> 57.9 | Helped |
| jointly train with ScanNet and ARKitScenes rather than only one source dataset (ablation) | UniDet3D target training · S3DIS | mAP50: 55.3 -> 57.9 | Helped |
| add positional encoding (ablation) | UniDet3D · ScanNet | mAP50: 66.1 -> 66.0 | No clear effect |
| replace disentangled matching with Hungarian matching (ablation) | UniDet3D · ScanNet | mAP50: 66.1 -> 64.5 | Hurt |
| increase transformer layers from 0 to 6 (ablation) | UniDet3D encoder · ScanNet | mAP50: 52.2 -> 65.2 | Helped |
| increase transformer layers from 6 to 9 (ablation) | UniDet3D encoder · ScanNet | mAP50: 65.2 -> 65.1 | No clear effect |
| replace Minkowski backend with SpConv (ablation) | UniDet3D sparse backbone · ScanNet | mAP50: 61.5 -> 65.2 | Helped |
Surprising findings
- Removing positional encoding and replacing Hungarian matching with disentangled matching improves ScanNet mAP50 slightly (66.1 vs 66.0 with PE only), indicating both transformer-specific components are redundant.
- Trained from scratch on small datasets, UniDet3D is notably inferior to convolutional baselines TR3D and FCAF3D.
Limitations
- Trained from scratch on small target datasets (S3DIS, MultiScan, 3RScan), the model underperforms convolutional baselines TR3D and FCAF3D.
- Evaluated only on indoor datasets with multi-view 3D reconstructions; single-view RGB-D datasets like SUN RGB-D were excluded.
Experimental setup
Datasets: ScanNet ARKitScenes S3DIS MultiScan 3RScan ScanNet++
Benchmarks: ScanNet ARKitScenes S3DIS MultiScan 3RScan ScanNet++
Metrics: mAP@0.25 mAP@0.5
Baselines: VoteNet GSDN FCAF3D TR3D CAGroup3D SPGroup3D V-DETR Uni3DETR 3DETR Group-Free Mask3D OneFormer3D
Builds on: OneFormer3D TR3D V-DETR Group-Free mmdetection3d
Closest papers by meaning
- Cubify Anything: Scaling Indoor 3D Object Detection (CVPR 2025 · similarity 0.90)
- One for All: Multi-Domain Joint Training for Point Cloud Based 3D Object Detection (NeurIPS 2024 · similarity 0.88)
- UnScene3D: Unsupervised 3D Instance Segmentation for Indoor Scenes (CVPR 2024 · similarity 0.88)
- Rooms from Motion: Un-posed Indoor 3D Object Detection as Localization and Mapping (NeurIPS 2025 · similarity 0.88)
- Uncertainty Meets Diversity: A Comprehensive Active Learning Framework for Indoor 3D Object Detection (CVPR 2025 · similarity 0.88)
- LabelAny3D: Label Any Object 3D in the Wild (NeurIPS 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.