Research map / Point clouds / Segmentation and detection
PSA-SSL: Pose and Size-aware Self-Supervised Learning on LiDAR Point Clouds
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Extends contrastive point-cloud SSL (SegContrast, DepthContrast) with a self-supervised bounding-box regression pretext task and LiDAR beam pattern augmentation to learn pose- and size-aware, sensor-agnostic LiDAR features.
Problem
Contrastive SSL on LiDAR point clouds learns features invariant to rotation, translation and scale, discarding object pose and size information needed for localization and geometry-sensitive 3D scene understanding, and transfers poorly across LiDAR sensors.
Approach
Add a bounding-box regression head (predicting per-point offsets to anchors from clustering-derived pseudo boxes) alongside the contrastive loss, and augment with LiDAR beam pattern transformations so that contrastive features become invariant to sensor pattern while retaining pose and size information.
Key results
- PSA-DC improves DepthContrast on 1% Waymo semantic segmentation from 50.36 to 53.02 mIoU; PSA-SC reaches 54.36 on 1% Waymo versus 53.50 for SegContrast.
- On SemanticKITTI with 0.1% labels PSA-SC underperforms BEVContrast (37.7 vs 39.7), but with 1% labels it reaches 55.3 mIoU vs 53.8 for BEVContrast.
- PSA-SC reduces SegContrast pretraining time per iteration by 33% (2.75 s vs 4.1 s).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add bounding box regression and LiDAR pattern augmentation to SegContrast | SegContrast · Waymo, nuScenes, and SemanticKITTI with 1% or 5% labels | mIoU: +0.86 to +2.39 | Helped |
| Add Single Pattern LiDAR augmentation (ablation) | SegContrast · Waymo, nuScenes, and SemanticKITTI, 1% labels | mIoU: 40.17 -> 43.44; 32.64 -> 34.88; 37.63 -> 39.09 | Helped |
| Add bounding box regression pretext task (ablation) | SegContrast · Waymo, nuScenes, and SemanticKITTI, 1% labels | mIoU: 40.17 -> 42.42; 32.64 -> 35.49; 37.63 -> 38.00 | Mixed |
| Use PolarMix instead of Single Pattern LiDAR augmentation (ablation) | SegContrast · Waymo, nuScenes, and SemanticKITTI, 1% labels | mIoU: 43.44 vs 40.96; 34.88 vs 33.74; 39.09 vs 38.34 | Hurt |
| Add Single Pattern LiDAR augmentation and bounding box regression together (ablation) | SegContrast · Waymo, nuScenes, and SemanticKITTI, 1% labels | mIoU: 40.17 -> 43.65; 32.64 -> 35.94; 37.63 -> 39.96 | Helped |
| Add bounding box regression and LiDAR pattern augmentation to DepthContrast | DepthContrast · Waymo, nuScenes, and SemanticKITTI, 1% labels | mIoU: +2.66; +2.43; +1.15 | Helped |
| Add bounding box regression and LiDAR pattern augmentation to SegContrast | SegContrast · Waymo 1% labels, object detection | 3D AP: +1.13/+1.03; -0.44/-0.36; -2.22/-2.14 | Mixed |
| Add bounding box regression and LiDAR pattern augmentation to SegContrast | SegContrast · KITTI 5% labels, moderate difficulty | 3D AP50 R40: 68.33 -> 68.86 | Helped |
| Add bounding box regression and LiDAR pattern augmentation to SegContrast | SegContrast · nuScenes 5% labels | mAP: 14.03 -> 14.73 | Helped |
| Add bounding box regression and LiDAR pattern augmentation to SegContrast | SegContrast · SemanticKITTI pretraining to nuScenes full | mIoU: +3.56 | Helped |
| Add LiDAR pattern augmentation and bounding box regression | SegContrast pretraining · Pretraining on Waymo | pretraining time: 33% reduced pretraining time | Helped |
Surprising findings
- Adding attention-based proposal encoding or scale/rotation regression to SegContrast gave performance similar to the baseline, while PSA gave significant gains.
- PSA-SC underperforms SegContrast on 5% KITTI small classes (pedestrian and cyclist), attributed to LiDAR pattern augmentation sparsifying small clusters and imperfect box targets.
Limitations
- Small object classes (pedestrians, cyclists) underperform in detection due to sparsified clusters and tight visible-part box targets.
- LiDAR pattern augmentation only transforms dense to sparse LiDARs, so it is omitted for nuScenes pretraining.
- Pretraining relies on clustering heuristics (Patchwork++, HDBSCAN) with a dataset-tuned epsilon parameter.
Experimental setup
Datasets: Waymo Open Dataset nuScenes SemanticKITTI KITTI
Benchmarks: Waymo Open Dataset nuScenes SemanticKITTI KITTI
Metrics: mIoU AP50 R40 mAP NDS
Baselines: DepthContrast SegContrast ProposalContrast-style attention encoder Shi and Rajkumar scale/rotation regression PointContrast ALSO BEVContrast TARL MAELi
Builds on: SegContrast DepthContrast MoCo LiDomAug PolarMix
Closest papers by meaning
- VLM2Scene: Self-Supervised Image-Text-LiDAR Learning with Foundation Models for Autonomous Driving Scene Understanding (AAAI 2024 · similarity 0.87)
- LiDAR-PTQ: Post-Training Quantization for Point Cloud 3D Object Detection (ICLR 2024 · similarity 0.87)
- Leveraging Imagery Data with Spatial Point Prior for Weakly Semi-supervised 3D Object Detection (AAAI 2024 · similarity 0.87)
- Mitigating Object Dependencies: Improving Point Cloud Self-Supervised Learning through Object Exchange (CVPR 2024 · similarity 0.87)
- Sonata: Self-Supervised Learning of Reliable Point Representations (CVPR 2025 · similarity 0.86)
- Seg2Box: 3D Object Detection by Point-Wise Semantics Supervision (AAAI 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.