Research map / Point clouds / Segmentation and detection
PDF: A Probability-Driven Framework for Open World 3D Point Cloud Semantic Segmentation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a Probability-Driven Framework (PDF) for open-world 3D point cloud semantic segmentation that adds a lightweight U-decoder for uncertainty estimation, a heuristic+graph-boundary pseudo-labeling scheme for unknown objects, and an incremental knowledge distillation strategy for adding novel classes.
Problem
Point cloud semantic segmentation networks are closed-set and static, so they cannot identify unknown classes or incrementally learn new ones; the paper targets open-world semantic segmentation (OSS plus incremental learning) on indoor 3D scenes.
Approach
Estimate uncertainty with a lightweight U-decoder supervised by pseudo labels generated from softmax uncertainty via a seed-based heuristic search (HUA) and a GMM-based graph boundary detection (GBD), and expand the model incrementally with KL-divergence distillation from the open-set teacher.
Key results
- On S3DIS with PointTransformer, PDF reaches 73.1 AUPR and 96.2 AUROC for unknown identification versus APF's 31.6 and 90.0, at 68.6 mIoU.
- On ScanNetv2 with StratifiedTransformer, PDF reaches 66.4 AUPR and 92.7 AUROC versus REAL's 50.9 and 87.1.
- In incremental learning on S3DIS, PDF achieves 69.4 mIoU, 64.3 mIoU_novel and 70.3 mIoU_old, above REAL's 68.9 / 61.3 / 70.3.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add the U-decoder instead of using an MLP layer (ablation) | PDF with StratifiedTransformer · ScanNetv2 | AUPR: 67.6 -> 67.8 | Helped |
| Retain semantic output OS in the joint loss (ablation) | PDF with StratifiedTransformer · ScanNetv2 | AUPR: 31.0 -> 67.8 | Helped |
| Add pseudo-labeling and loss LP (ablation) | PDF with StratifiedTransformer · ScanNetv2 | AUPR: 20.0 -> 67.8 | Helped |
| Retain the GBD algorithm (ablation) | PDF with StratifiedTransformer · ScanNetv2 | AUPR: 64.3 -> 67.8 | Helped |
| Use PDF rather than REAL for incremental learning | PointTransformer · S3DIS incremental learning | mIoUnovel: 61.3 -> 64.3 | Helped |
| Use PDF rather than REAL for incremental learning | PointTransformer · S3DIS incremental learning | mIoUold: 70.3 -> 70.3 | No clear effect |
| Use PDF rather than REAL for open-set segmentation | PointTransformer · S3DIS OSS | AUPR: 25.4* -> 73.1 | Helped |
| Use PDF rather than REAL for open-set segmentation | StratifiedTransformer · ScanNetv2 OSS | AUPR: 58.1 -> 67.8 | Helped |
Surprising findings
- MSP and MaxLogit baselines achieve the highest closed-set mIoU because they do not modify the network, with PDF trading a slight closed-set mIoU drop for better unknown identification.
- Removing the semantic output from the U-decoder supervision (Model B) collapses AUPR to 31.0 on ScanNetv2, showing the joint supervision is critical.
Limitations
- Performance is constrained on outdoor scenes where sparse, incomplete objects are misclassified with high confidence, per the authors' conclusion.
- Evaluated only on indoor S3DIS and ScanNetv2; APF results on S3DIS are copied from the original paper because its code is not released.
Experimental setup
Datasets: S3DIS ScanNetv2
Benchmarks: S3DIS ScanNetv2
Metrics: mIoU AUROC AUPR mIoU_old mIoU_novel
Baselines: MSP MaxLogit MC-Dropout DMLNet REAL APF Finetune Feature extraction LwF
Builds on: REAL APF Learning without Forgetting MSP baseline (Hendrycks & Gimpel) PointTransformer
Closest papers by meaning
- Geometry and Uncertainty-Aware 3D Point Cloud Class-Incremental Semantic Segmentation (CVPR 2023 · similarity 0.91)
- LogoSP: Local-global Grouping of Superpoints for Unsupervised Semantic Segmentation of 3D Point Clouds (CVPR 2025 · similarity 0.88)
- Geometrically-driven Aggregation for Zero-shot 3D Point Cloud Understanding (CVPR 2024 · similarity 0.87)
- Weakly Supervised Point Cloud Semantic Segmentation via Artificial Oracle (CVPR 2024 · similarity 0.87)
- Open-World Semantic Segmentation Including Class Similarity (CVPR 2024 · similarity 0.87)
- Generalized Zero-Shot Learning for Point Cloud Segmentation with Evidence-Based Dynamic Calibration (AAAI 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.