Research map / Point clouds / Segmentation and detection
GroupContrast: Semantic-aware Self-supervised Representation Learning for 3D Understanding
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes GroupContrast, a self-supervised 3D scene pre-training method that combines prototype-based segment grouping with semantic-aware, confidence-weighted InfoNCE contrastive learning to reduce "semantic conflict" among false negatives.
Problem
Point-discrimination pretext tasks for self-supervised 3D scene representation learning treat unmatched points as negatives even when they are semantically identical, creating false negatives and "semantic conflict" that hurts downstream dense prediction.
Approach
Segment grouping clusters geometric segments into semantic groups via a teacher-student prototype assignment (DINO-style centring and sharpening, entropy-weighted distillation); points in the same group become positives in a confidence-weighted InfoNCE loss, so semantically similar points are no longer pushed apart.
Key results
- Achieves 75.7% mIoU on ScanNet and 30.0% mIoU on ScanNet200 semantic segmentation, outperforming prior SOTA (MSC) by 0.7 and 1.2 points.
- Reaches 62.3 mAP@0.5 on ScanNet instance segmentation, 2.7 points above the previous best self-supervised method.
- Reaches 41.1 mAP@0.5 on ScanNet object detection and 37.0 on SUN RGB-D, surpassing prior SOTA by 1.8 and 0.6 points.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use Segment Grouping rather than matched-point positive pairs (ablation) | GroupContrast · ScanNet semantic segmentation | FT mIoU (%): 74.6 -> 75.7 | Helped |
| Use spatial grids for positive pairs instead of matched points (ablation) | GroupContrast · ScanNet semantic segmentation | FT mIoU (%): 74.6 -> 74.2 | Hurt |
| Use geometry segments for positive pairs instead of matched points (ablation) | GroupContrast · ScanNet semantic segmentation | FT mIoU (%): 74.6 -> 74.8 | No clear effect |
| Set the prototype count to 32 rather than 16, 64, or 128 (ablation) | GroupContrast · ScanNet semantic segmentation | FT mIoU (%): 16: 75.2; 32: 75.7; 64: 75.3; 128: 74.8 | Helped |
| Add informative-aware distillation weights (ablation) | GroupContrast · ScanNet semantic segmentation | FT mIoU (%): 75.4 -> 75.7 | Helped |
| Set teacher temperature τt to 0.07 rather than 0.04 (ablation) | GroupContrast · ScanNet semantic segmentation | FT mIoU (%): 75.3 -> 75.7 | Helped |
| Add the asymmetric predictor (ablation) | GroupContrast · ScanNet semantic segmentation | FT mIoU (%): 75.3 -> 75.7 | Helped |
| Add semantic-aware positive pairs (ablation) | GroupContrast · ScanNet semantic segmentation | FT mIoU (%): 74.8 -> 75.1 | Helped |
| Add confidence weights to semantic-aware positive pairs (ablation) | GroupContrast · ScanNet semantic segmentation | FT mIoU (%): 75.1 -> 75.7 | Helped |
| Add centering and sharpening to prevent collapse (ablation) | GroupContrast · ScanNet semantic segmentation | FT mIoU (%): 72.7 -> 75.7 | Helped |
| Increase pre-training epochs from 600 to 1200 (ablation) | GroupContrast · ScanNet semantic segmentation | FT mIoU (%): 75.7 -> 75.7 | No clear effect |
| Increase pre-training epochs from 600 to 1200 (ablation) | GroupContrast · ScanNet (20%) semantic segmentation | FT mIoU (%): 65.8 -> 66.5 | Helped |
Surprising findings
- Pre-training for 1200 epochs does not improve full-data ScanNet semantic segmentation over 600 epochs (both 75.7), though it helps in the 20% data-efficient setting.
Limitations
- Pre-training data limited to the relatively small ScanNet scale.
- Grouping labels are unreliable in early training, requiring confidence weighting and centring/sharpening to avoid collapse.
Experimental setup
Datasets: ScanNet ScanNet200 S3DIS SUN RGB-D
Benchmarks: ScanNet semantic segmentation ScanNet200 semantic segmentation S3DIS semantic segmentation ScanNet instance segmentation ScanNet200 instance segmentation S3DIS instance segmentation ScanNet object detection SUN RGB-D object detection ScanNet data-efficient segmentation
Metrics: mIoU mAP@0.5
Baselines: PointContrast CSC MSC Cluster3Dseg Scratch (SC)
Builds on: PointContrast DINO InfoNCE Felzenszwalb graph-based segmentation Masked Scene Contrast
Closest papers by meaning
- Self-Supervised Visual Representation Learning with Semantic Grouping (NeurIPS 2022 · similarity 0.89)
- LogoSP: Local-global Grouping of Superpoints for Unsupervised Semantic Segmentation of 3D Point Clouds (CVPR 2025 · similarity 0.87)
- MESC-3D:Mining Effective Semantic Cues for 3D Reconstruction from a Single Image (CVPR 2025 · similarity 0.86)
- Building a Strong Pre-Training Baseline for Universal 3D Large-Scale Perception (CVPR 2024 · similarity 0.86)
- Sonata: Self-Supervised Learning of Reliable Point Representations (CVPR 2025 · similarity 0.86)
- A Unified Framework for 3D Scene Understanding (NeurIPS 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.