Research map / Point clouds / Segmentation and detection
FreePoint: Unsupervised Point Cloud Instance Segmentation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes FreePoint, an unsupervised class-agnostic 3D instance segmentation framework that builds pseudo masks via RAMA multicut over coordinate, color and self-supervised features, then trains Mask3D with a weakly-supervised loss and two-step training.
Problem
Instance segmentation of point clouds normally requires heavy per-point or box annotations; the paper targets unsupervised class-agnostic instance segmentation of indoor point clouds without any annotations.
Approach
Combines coordinates, colors and self-supervised deep features into point affinities, runs a bottom-up GPU multicut (RAMA) repeatedly with an id-as-feature ensemble to produce coarse pseudo masks, and trains Mask3D with center/bounding-box losses in a two-step base-then-under-segmented schedule with an undersegmentation-ignore matching rule.
Key results
- On ScanNet validation, FreePoint reaches 18.9 AP / 36.4 AP50 for unsupervised class-agnostic instance segmentation, versus 13.3 AP for UnScene3D on the train split (FreePoint 21.4 AP on train).
- As a pretext task on S3DIS with 10% masks, FreePoint reaches 42.9 AP versus 34.7 for training from scratch and 37.1 for CSC.
- Ablation: removing the weakly-supervised center and box losses drops AP from 18.9 to 14.3 and 15.2 respectively.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add deep features alongside coordinates and colors (ablation) | FreePoint point feature representation · ScanNet final instance segmentation | AP: 16.5 -> 18.9 | Helped |
| use 10 RAMA runs for id-as-feature ensembling instead of a single run (ablation) | FreePoint pseudo-mask generation · ScanNet final instance segmentation | AP: 13.8 -> 18.9 | Helped |
| add id-as-feature ensembling with 5 RAMA runs (ablation) | FreePoint pseudo-mask generation · ScanNet final instance segmentation | AP: 13.8 -> 17.6 | Helped |
| add Lmean (ablation) | FreePoint weakly-supervised training · ScanNet | AP: 14.3 -> 18.9 | Helped |
| add Lbox (ablation) | FreePoint weakly-supervised training · ScanNet | AP: 15.2 -> 18.9 | Helped |
| add Lmean and Lbox (ablation) | FreePoint weakly-supervised training · ScanNet | AP: 14.0 -> 18.9 | Helped |
| add Dice and BCE loss terms (ablation) | FreePoint weakly-supervised training · ScanNet | AP: 7.8 -> 18.9 | Helped |
| add undersegmentation-ignore (ablation) | FreePoint training · ScanNet | AP: 16.8 -> 18.9 | Helped |
| use a two-step schedule: base masks followed by under-segmented masks, rather than either alone (ablation) | FreePoint training · ScanNet | AP: 14.2 -> 18.9 | Helped |
| use under-segmented masks alone instead of the two-step schedule (ablation) | FreePoint training · ScanNet | AP: 18.9 -> 6.4 | Hurt |
| use FreePoint pre-training rather than CSC pre-training | S3DIS fine-tuning with 10% masks · S3DIS, 10% masks | AP: 37.1 -> 42.9 | Helped |
| use FreePoint pre-training rather than CSC pre-training | S3DIS fine-tuning with 10% scenes · S3DIS, 10% scenes | AP: 32.7 -> 37.2 | Helped |
Surprising findings
- FreeMasks and spectral methods, successful in 2D unsupervised segmentation, fail to transfer to cluttered point cloud scenes.
- Training only with the weakly-supervised losses without dice/BCE gives much worse results (7.8 AP) than fully-supervised losses alone.
Limitations
- Relies on plane segmentation to remove backgrounds, tailored to indoor scenes with large flat surfaces.
- Pseudo masks are noisy; the method depends on multicut randomness mitigated by repeated runs.
Experimental setup
Datasets: ScanNet S3DIS
Benchmarks: ScanNet S3DIS
Metrics: AP AP50 AP25
Baselines: DBSCAN HDBSCAN Nunes et al. UnScene3D Supervoxel FreeMasks 3D Spectral PointContrast DepthContrast CSC
Builds on: RAMA Mask3D CSC UnScene3D FreeMasks
Closest papers by meaning
- DyCo3D: Robust Instance Segmentation of 3D Point Clouds Through Dynamic Convolution (CVPR 2021 · similarity 0.91)
- Towards Foundation Models for 3D Scene Understanding: Instance-Aware Self-Supervised Learning for Point Clouds (CVPR 2026 · similarity 0.90)
- PointGroup: Dual-Set Point Grouping for 3D Instance Segmentation (CVPR 2020 · similarity 0.90)
- Spherical Mask: Coarse-to-Fine 3D Point Cloud Instance Segmentation with Spherical Representation (CVPR 2024 · similarity 0.89)
- OneFormer3D: One Transformer for Unified Point Cloud Segmentation (CVPR 2024 · similarity 0.88)
- Mining and Transferring Feature-Geometry Coherence for Unsupervised Point Cloud Registration (NeurIPS 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.