Research map / Detection and segmentation / 3D detection and driving
Edge-Aware 3D Instance Segmentation Network with Intelligent Semantic Prior
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes EASE, a transformer-based 3D instance segmentation framework that injects CLIP text embeddings of predicted classes into instance queries and adds a voxel-wise edge prediction module supervised by KNN-derived pseudo edge labels.
Problem
Transformer-based 3D instance segmentation relies mainly on geometry, so it confuses instances with similar appearance but different semantic roles and produces ambiguous, fuzzy instance edges.
Approach
Fuse CLIP text embeddings of each query's predicted category into instance queries via a Semantic Fusion network applied across decoder layers, and add an auxiliary edge prediction head whose targets are pseudo edge labels computed from point-wise instance labels with a KNN label-disagreement rule.
Key results
- On ScanNetV2 validation, EASE reaches 60.2 mAP / 77.2 mAP50, versus 59.9 / 76.5 for MAFT.
- On ScanNet200 validation, EASE reaches 29.9 mAP, versus 29.2 for MAFT.
- Ablation on S3DIS Area 5 shows the semantic network adds +1.5 mAP and the edge module adds +1.9 mAP, with both giving +2.4 mAP over the baseline.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add the Semantic Network (ablation) | EASE · ScanNetV2 val | mAP: 58.4 -> 59.3 | Helped |
| Add the Semantic Network (ablation) | EASE · S3DIS Area 5 | mAP: 56.6 -> 58.5 | Helped |
| Add the Edge Prediction Module (ablation) | EASE · ScanNetV2 val | mAP: 58.4 -> 59.3 | Helped |
| Add the Edge Prediction Module (ablation) | EASE · S3DIS Area 5 | mAP: 56.6 -> 58.5 | Helped |
| Add both Semantic Network and Edge Prediction Module (ablation) | EASE · ScanNetV2 val | mAP: 58.4 -> 60.2 | Helped |
| Add both Semantic Network and Edge Prediction Module (ablation) | EASE · S3DIS Area 5 | mAP: 56.6 -> 59.0 | Helped |
| Use text embeddings in fcls with element-wise sum (ablation) | EASE semantic guidance · ScanNetV2 val | mAP: 58.4 -> 58.9 | Helped |
| Use text embeddings in fcls with concatenation (ablation) | EASE semantic guidance · ScanNetV2 val | mAP: 58.4 -> 59.3 | Helped |
| Apply Semantic Fusion with element-wise sum (ablation) | EASE semantic guidance · ScanNetV2 val | mAP: 58.4 -> 59.6 | Helped |
| Apply Semantic Fusion with concatenation (ablation) | EASE semantic guidance · ScanNetV2 val | mAP: 58.4 -> 60.2 | Helped |
| Predict edges from high-resolution F0 rather than low-resolution F4 (ablation) | EASE edge prediction · ScanNetV2 val | mAP: 59.3 vs. 59.0 | Helped |
| Use edge-loss weight w=10 rather than w=2 (ablation) | EASE edge prediction · S3DIS Area 5 | mAP: 56.9 / 56.7 -> 58.5 / 58.2 | Helped |
Limitations
- Gains over the strongest prior method (MAFT) are small on ScanNetV2 test (59.8 vs 59.6 mAP) and ScanNet200 (about +0.7 mAP).
- Edge loss weight w must be tuned per dataset scale, and the optimal value varies with scene size.
Experimental setup
Datasets: ScanNetV2 ScanNet200 S3DIS STPLS3D
Benchmarks: ScanNetV2 ScanNet200 S3DIS STPLS3D
Metrics: mAP mAP50 mAP25 mPrec mRec
Baselines: Mask3D MAFT QueryFormer SPFormer SoftGroup HAIS PointGroup SSTNet
Builds on: Mask3D MAFT CLIP SPFormer
Closest papers by meaning
- Geometry-Guided Semantic Reconstruction for 3D Instance Segmentation (NeurIPS 2026 · similarity 0.89)
- Insightful Instance Features for 3D Instance Segmentation (CVPR 2025 · similarity 0.89)
- Segment Any 3D Object with Language (ICLR 2025 · similarity 0.88)
- SAI3D: Segment Any Instance in 3D Scenes (CVPR 2024 · similarity 0.87)
- InstanceGaussian: Appearance-Semantic Joint Gaussian Representation for 3D Instance-Level Perception (CVPR 2025 · similarity 0.87)
- Open3DIS: Open-Vocabulary 3D Instance Segmentation with 2D Mask Guidance (CVPR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.