Research map / Detection and segmentation / 3D detection and driving
Learning Class Prototypes for Unified Sparse-Supervised 3D Object Detection
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes CPDet3D, a unified sparse-supervised (one object per scene) 3D object detector for indoor and outdoor scenes that mines unlabeled objects via optimal-transport matching to learned class prototypes and refines with pseudo and prototype labels.
Problem
Sparse-supervised 3D object detection relies on GT sampling to fill missing categories in each scene, which fails for indoor scenes with scene-specific categories; existing methods are outdoor-only and leave many objects unannotated.
Approach
Clusters labeled object features across scenes into class-aware prototypes, matches unlabeled features to prototypes via Sinkhorn-Knopp optimal transport to assign prototype labels beyond single-scene limits, and combines these with iteratively filtered pseudo labels in a multi-label cooperative refinement stage.
Key results
- ScanNet V2 one-object-per-scene: 56.1 mAP@0.25 and 40.8 mAP@0.5 vs TR3D fully supervised 72.0/57.4 (about 78%), and SparseDet 46.0/28.2.
- KITTI Car-3D AP moderate 82.2 vs CoIn++ 79.5; Car-BEV moderate 91.8 vs 88.8, reaching about 96% of fully supervised performance.
- SUN RGB-D mAP@0.25 60.2 and mAP@0.5 43.3 under one object per scene, about 90% of fully supervised; ScanNet V2 54.6/36.6 beats DQS3D semi-supervised (49.2/35.0).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add CPDet3D prototype mining and multi-label refinement | TR3D · ScanNet V2 val | mAP@0.25: 46.0 -> 56.1 | Helped |
| add CPDet3D prototype mining and multi-label refinement | TR3D · SUN RGB-D val | mAP@0.25: 56.7 -> 60.2 | Helped |
| add CPDet3D prototype mining and multi-label refinement | Voxel-RCNN · KITTI val, Car-3D AP R40 Moderate | Car-3D AP R40 Moderate: 79.5 -> 82.2 | Helped |
| add CPDet3D prototype mining and multi-label refinement | Voxel-RCNN · KITTI val, Car-BEV AP R40 Moderate | Car-BEV AP R40 Moderate: 88.8 -> 91.8 | Helped |
| add PLM using classification-score label matching (ablation) | TR3D on ScanNet V2 one object per scene · ScanNet V2 one object per scene | mAP@0.25: 37.6 to 38.0 | Helped |
| add class-aware prototype clustering (CPC) (ablation) | TR3D + PLM on ScanNet V2 one object per scene · ScanNet V2 one object per scene | mAP@0.25: 38.0 to 39.3 | Helped |
| add class-aware prototype clustering (CPC) (ablation) | TR3D + PLM on ScanNet V2 one object per scene · ScanNet V2 one object per scene | mAP@0.5: 23.6 to 26.5 | Helped |
| add multi-label cooperative refinement (MCR) with iterative pseudo labeling (ablation) | TR3D on ScanNet V2 one object per scene · ScanNet V2 one object per scene | mAP@0.25: 37.6 to 51.9 | Helped |
| add prototype labels (PLM) to sparse and pseudo labels in MCR (ablation) | TR3D + MCR on ScanNet V2 one object per scene · ScanNet V2 one object per scene | mAP@0.5: 32.3 to 38.3 | Helped |
| combine sparse, prototype, and pseudo labels (ablation) | CPDet3D label mining on ScanNet V2 training set · ScanNet V2 training set | mAR: 8.3; 47.8; 33.4; 67.1 | Helped |
| vary the number of prototypes per class; 10 is best (ablation) | CPDet3D on ScanNet V2 · ScanNet V2 | mAP@0.25: 37.4; 39.1; 39.3; 37.6 | Mixed |
| set prototype warm-up iterations to 1000 (ablation) | CPDet3D on ScanNet V2 · ScanNet V2 | mAP@0.25: 37.8; 39.3; 38.2; 39.1 | Helped |
Surprising findings
- Prototype labels have only about 71.1% precision, yet adding them still improves recall and detection (mAR 33.4 to 67.1 with all labels).
Limitations
- Prototype labels are noisy (about 71% precision), relying on NMS and iterative training to offset errors.
- Hyperparameters such as warm-up iterations, prototype count, and thresholds were tuned on the evaluation setting.
Experimental setup
Datasets: ScanNet V2 SUN RGB-D KITTI
Benchmarks: ScanNet V2 SUN RGB-D KITTI
Metrics: mAP@0.25 mAP@0.5 3D AP R40 BEV AP R40 mAR
Baselines: TR3D (sparse) Co-mining SparseDet CoIn CoIn++ SS3D SESS IoUMatch DKPE Diff-SS3D Diff3DETR DQS3D
Builds on: TR3D Voxel-RCNN CoIn MixSup Sinkhorn-Knopp
Closest papers by meaning
- FSHNet: Fully Sparse Hybrid Network for 3D Object Detection (CVPR 2025 · similarity 0.84)
- UNION: Unsupervised 3D Object Detection using Object Appearance-based Pseudo-Classes (NeurIPS 2024 · similarity 0.83)
- HINTED: Hard Instance Enhanced Detector with Mixed-Density Feature Fusion for Sparsely-Supervised 3D Object Detection (CVPR 2024 · similarity 0.83)
- SP3D: Boosting Sparsely-Supervised 3D Object Detection via Accurate Cross-Modal Semantic Prompts (CVPR 2025 · similarity 0.83)
- SparseAlign: a Fully Sparse Framework for Cooperative Object Detection (CVPR 2025 · similarity 0.83)
- SAFDNet: A Simple and Effective Network for Fully Sparse 3D Object Detection (CVPR 2024 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.