Research map / Detection and segmentation / 3D detection and driving
MixSup: Mixed-grained Supervision for Label-efficient LiDAR-based 3D Object Detection
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes MixSup, which trains LiDAR 3D detectors with cheap cluster-level semantic labels plus a small fraction of accurate box labels, and PointSAM, which uses SAM to auto-generate those cluster labels.
Problem
Label-efficient LiDAR-based 3D object detection: reducing annotation cost while keeping accuracy, given that point clouds are textureless so semantics are hard to learn but geometry is easy to learn from few labels.
Approach
Mixed-grained supervision: massive cheap cluster-level labels supervise classification/confidence, while a few box-level labels supervise regression; label assignment (center-based and box-based) is redesigned to use cluster centers and box-cluster IoU so mainstream detectors can be plugged in. PointSAM generates coarse clusters from SAM masks refined by connected-component labeling.
Key results
- MixSup with CenterPoint reaches 97.31% of fully supervised mAP on nuScenes (60.73 vs 62.41) using cluster labels and 10% boxes.
- On Waymo Open Dataset, SST with MixSup achieves 60.74 mean L2 mAPH at 10% annotation cost, versus 50.46 for the MV-JAR semi-supervised baseline.
- PointSAM reaches 63.7 PQ on nuScenes thing classes, comparable to fully supervised panoptic models such as GP-S3Net (56.0) and Panoptic-PolarNet (59.2), though below CFNet (74.8).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use MixSup with coarse cluster labels and 10% accurate box labels instead of training on 10% of frames | CenterPoint · nuScenes val | mAP: 42.19 -> 60.73 | Helped |
| Use MixSup with coarse cluster labels and 10% accurate box labels instead of training on 10% of frames | CenterPoint · Waymo Open Dataset val | L2 APH Mean: 51.64 -> 62.34 | Helped |
| Replace cluster-level labels with fitted pseudo boxes, ignoring shape and heading supervision (ablation) | MixSup · nuScenes val, 10 epochs | mAP: 59.48 -> 55.75 | Hurt |
| Replace cluster-level labels with fitted pseudo boxes, ignoring heading supervision (ablation) | MixSup · nuScenes val, 10 epochs | mAP: 59.48 -> 56.22 | Hurt |
| Add center coordinates to coarse cluster supervision (ablation) | MixSup · nuScenes val, 10 epochs | mAP: 59.48 -> 60.49 | Helped |
| Add shape dimensions after adding center coordinates (ablation) | MixSup + Center · nuScenes val, 10 epochs | mAP: 60.49 -> 60.79 | Helped |
| Add heading after adding center coordinates and shape dimensions (ablation) | MixSup + Center + Shape · nuScenes val, 10 epochs | mAP: 60.79 -> 60.95 | Helped |
| Replace cluster-to-box IoU with box-to-box IoU for second-stage matching (ablation) | MixSup with FSD · Waymo Open Dataset val | L2 mAPH Mean: 68.57 -> 68.96 | No clear effect |
| Remove false-positive clusters from PointSAM labels (ablation) | CenterPoint with MixSup · nuScenes val | mAP: 49.49 -> 53.09 | Helped |
| Add false-negative labels after removing false-positive clusters (ablation) | CenterPoint with MixSup · nuScenes val | mAP: 53.09 -> 58.30 | Helped |
| Add simple self-training to MixSup (ablation) | SECOND · KITTI val moderate, Pedestrian | mAP: 50.18 -> 56.89 | Helped |
| Add simple self-training to MixSup (ablation) | PV-RCNN · KITTI val moderate, Cyclist | mAP: 65.67 -> 70.91 | Helped |
Surprising findings
- Pilot study shows detectors trained on 5% of well-classified crops perform nearly as well as 100%, suggesting few accurate boxes suffice for geometry.
- Oracle experiment using true box-to-box IoU instead of box-cluster IoU gives no significant gain, indicating the ambiguity of box-cluster IoU is not harmful.
Limitations
- Box-cluster IoU is ambiguous, and handcrafted box fitting from clusters performs poorly for large objects like Car and Truck.
- PointSAM produces false-positive clusters and false negatives from projection and calibration errors, and needs human rectification to reach the reported results.
- Integration with semi-supervised methods is only briefly explored via simple self-training.
Experimental setup
Datasets: nuScenes Waymo Open Dataset KITTI nuImages
Benchmarks: nuScenes Waymo Open Dataset KITTI
Metrics: mAP NDS L2 APH 3D AP PQ SQ RQ
Baselines: WS3D FixMatch ProficientTeacher MV-JAR HSSDA 3DIoUMatch GP-S3Net CFNet L-shape fitting
Builds on: SECOND CenterPoint PV-RCNN FSD Segment Anything Model
Closest papers by meaning
- Seg2Box: 3D Object Detection by Point-Wise Semantics Supervision (AAAI 2025 · similarity 0.88)
- SeMoLi: What Moves Together Belongs Together (CVPR 2024 · similarity 0.88)
- Semi-supervised 3D Object Detection with PatchTeacher and PillarMix (AAAI 2024 · similarity 0.88)
- iDet3D: Towards Efficient Interactive Object Detection for LiDAR Point Clouds (AAAI 2024 · similarity 0.88)
- Leveraging Imagery Data with Spatial Point Prior for Weakly Semi-supervised 3D Object Detection (AAAI 2024 · similarity 0.87)
- A-Teacher: Asymmetric Network for 3D Semi-Supervised Object Detection (CVPR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.