Research map / Point clouds / Segmentation and detection
Relation3D : Enhancing Relation Modeling for Point Cloud Instance Segmentation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Relation3D improves transformer-based 3D point cloud instance segmentation by modeling internal scene-feature and query relations via adaptive superpoint aggregation, contrastive-guided superpoint refinement, and relation-aware self-attention built on Maft.
Problem
Transformer-based 3D instance segmentation models capture external scene-query relations via mask attention but neglect internal relations among superpoint (scene) features and among instance queries, and their position embeddings lack precise spatial meaning.
Approach
Three modules: (1) adaptive superpoint aggregation that learns per-point softmax weights (max and mean pooling) instead of plain pooling; (2) a dual-path superpoint refinement where superpoint features cross-attend to queries, supervised by a BCE contrastive loss on pairwise same-instance relations; (3) relation-aware self-attention that adds positional and geometric log-relative relations between
Key results
- On ScanNetV2 validation, Relation3D reaches 62.5 mAP, 80.2 AP@50 and 87.0 AP@25, versus 58.4 mAP for Maft.
- On ScanNetV2 hidden test, it reaches 62.2 mAP, versus 57.8 for Maft.
- Ablation on ScanNetV2 validation: ASAM, CLSR and RSA each add gains over the Maft baseline (59.8 mAP), with the full model at 62.5 mAP.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use Relation3D instead of Spherical Mask | ScanNetV2 validation · ScanNetV2 validation | mAP: 62.3 -> 62.5 | Helped |
| use Relation3D instead of Spherical Mask | ScanNetV2 test · ScanNetV2 test | mAP: 61.6 -> 62.2 | Helped |
| use Relation3D instead of Maft | ScanNet++ · ScanNet++ validation | mAP: 23.1 -> 28.2 | Helped |
| use Relation3D instead of Maft | ScanNet++ · ScanNet++ test | mAP: 20.9 -> 24.2 | Helped |
| use Relation3D instead of Maft | ScanNet200 · ScanNet200 validation | mAP: 29.2 -> 31.6 | Helped |
| use Relation3D instead of Spherical Mask | S3DIS Area5 · S3DIS Area5 | AP@50: 72.3 -> 72.5 | Helped |
| add ASAM (ablation) | Maft baseline · ScanNetV2 validation | mAP: 59.8 -> 60.1 | Helped |
| add CLSR (ablation) | Maft baseline · ScanNetV2 validation | mAP: 59.8 -> 60.9 | Helped |
| add ASAM and CLSR (ablation) | Maft baseline · ScanNetV2 validation | mAP: 59.8 -> 61.5 | Helped |
| add RSA (ablation) | Maft baseline · ScanNetV2 validation | mAP: 59.8 -> 61.0 | Helped |
| add ASAM, CLSR, and RSA (ablation) | Maft baseline · ScanNetV2 validation | mAP: 59.8 -> 62.5 | Helped |
| add contrastive loss to ASAM (ablation) | ASAM · ScanNetV2 validation | mAP: 60.1 -> 60.5 | Helped |
Limitations
- Gains over Maft are modest on some benchmarks (e.g., ScanNet200 mAP 31.6 vs 29.2) and the paper reports no variance across runs.
- Relies on superpoints and instance annotations to build the same-instance contrastive target.
- Built on Maft with additional modules, so added decoder cost and complexity are not fully quantified.
Experimental setup
Datasets: ScanNetV2 ScanNet++ ScanNet200 S3DIS
Benchmarks: ScanNetV2 ScanNet++ ScanNet200 S3DIS
Metrics: mAP AP@50 AP@25 Box AP@50 Box AP@25
Baselines: Maft Mask3D QueryFormer SPFormer SoftGroup Spherical Mask ISBNet PointGroup
Builds on: Maft Mask3D SPFormer Relation-DETR Sparse UNet
Closest papers by meaning
- OneFormer3D: One Transformer for Unified Point Cloud Segmentation (CVPR 2024 · similarity 0.88)
- FreePoint: Unsupervised Point Cloud Instance Segmentation (CVPR 2024 · similarity 0.87)
- PointCFormer: A Relation-Based Progressive Feature Extraction Network for Point Cloud Completion (AAAI 2025 · similarity 0.87)
- Insightful Instance Features for 3D Instance Segmentation (CVPR 2025 · similarity 0.87)
- RelationVGGT : Visual Geometry Transformers for 3D Spatial Relation Segmentation (NeurIPS 2026 · similarity 0.87)
- Sonata: Self-Supervised Learning of Reliable Point Representations (CVPR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.