Research map / Detection and segmentation / Weakly supervised detection and segmentation
Leveraging Imagery Data with Spatial Point Prior for Weakly Semi-supervised 3D Object Detection
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Point-DETR3D, a teacher-student framework for weakly semi-supervised 3D object detection from LiDAR with point annotations, adding explicit positional query initialization, image-LiDAR deformable RoI fusion, and point-guided feature-invariant self-supervision.
Problem
Weakly semi-supervised 3D object detection using only one point annotation per instance plus a few fully labeled scenes; adapting Point-DETR to 3D yields weak positional priors and poor pseudo-labels for distant, sparsely observed objects.
Approach
Initialize object queries' reference points directly from GT point annotations (explicit positional query initialization), fuse dense camera RoI features with BEV LiDAR RoI features via point-centric deformable RoI cross-attention, and apply point-weighted feature contrastive learning in the student.
Key results
- Teacher with 2% labeled data reaches 80.47 mAP / 66.03 SPNDS (pillar) vs Point-DETR 23.55 mAP / 23.87 SPNDS.
- With 5% labeled data, the student (CenterPoint-Voxel) reaches 53.99 SPNDS / 50.80 mAP vs 61.80 / 56.37 for the fully supervised model, i.e. about 90% performance.
- At distances over 40m, Point-DETR3D improves over Point-DETR by 105% SPNDS and 129% mAP.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace Point-DETR with Point-DETR3D | Object-DGCNN teacher · nuScenes val, Pillar-based, 2% labeled | SPNDS: 23.87 -> 66.03 | Helped |
| Replace Point-DETR with Point-DETR3D | Object-DGCNN teacher · nuScenes val, Voxel-based, 5% labeled | mAP: 44.56 -> 83.63 | Helped |
| Replace Point-DETR with Point-DETR3D | CenterPoint-Pillar student · nuScenes val, 5% labeled | SPNDS: 35.14 -> 45.81 | Helped |
| Replace Point-DETR with Point-DETR3D | CenterPoint-Voxel student · nuScenes val, 5% labeled | mAP: 39.26 -> 50.80 | Helped |
| Use explicit rather than implicit positional query initialization (ablation) | Object-DGCNN teacher · nuScenes val | mAP: 57.58 -> 68.67 (+11.09) | Helped |
| Add Point-Centric Deformable RoI cross-modal fusion (ablation) | Object-DGCNN teacher · nuScenes val | SPNDS: 70.26 -> 76.46 | Helped |
| Add Point-Centric Deformable RoI cross-modal fusion (ablation) | Object-DGCNN teacher · nuScenes val | mAP: 79.88 -> 87.46 | Helped |
| Add unconstrained full-feature-space self-supervised learning without point guidance (ablation) | CenterPoint-Voxel student · nuScenes val | mAP: 52.04 -> 51.85 (-0.19) | Hurt |
| Add point-guided reweighting to self-supervised feature learning (ablation) | CenterPoint-Voxel student · nuScenes val | mAP: 51.85 -> 53.03 (+0.99) | Helped |
| Train FocalFormer3D student without self-supervised learning (ablation) | FocalFormer3D · nuScenes val, 5% labeled | SPNDS/mAP | No clear effect |
Surprising findings
- Simply applying feature-level contrastive learning over the full BEV feature map slightly decreases performance (-0.19 mAP), so point-guided masking is needed.
- The pillar-based and voxel-based teachers perform similarly, suggesting saturation with point annotations alone.
Limitations
- Evaluated only on nuScenes with a single point annotation setting and fixed sampling of labeled ratios.
- Relies on camera calibration and paired multi-view images for the imagery branch.
- Some tables report results on the validation set, and the teacher's reported gains are large relative to baselines that may be less tuned.
Experimental setup
Datasets: nuScenes
Benchmarks: nuScenes
Metrics: SPNDS mAP
Baselines: Point-DETR CenterPoint TransFusion AutoAlignV2 MoCa FocalFormer3D
Builds on: Point-DETR Object-DGCNN CenterPoint Mean Teacher
Closest papers by meaning
- V-DETR: DETR with Vertex Relative Position Encoding for 3D Object Detection (ICLR 2024 · similarity 0.89)
- Semi-supervised 3D Object Detection with PatchTeacher and PillarMix (AAAI 2024 · similarity 0.89)
- Leveraging Temporal Cues for Semi-Supervised Multi-View 3D Object Detection (CVPR 2025 · similarity 0.88)
- Weakly Supervised Monocular 3D Detection with a Single-View Image (CVPR 2024 · similarity 0.88)
- LOCATE 3D: Real-World Object Localization via Self-Supervised Learning in 3D (ICML 2025 · similarity 0.88)
- A-Teacher: Asymmetric Network for 3D Semi-Supervised Object Detection (CVPR 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.