Research map / Point clouds / Segmentation and detection
Generative Hard Example Augmentation for Semantic Point Cloud Segmentation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes GHEA, which uses a conditional generative (VAE-style) network to reshape semantic point clouds into new labeled examples and mixes them by segmentation error to create hard training examples for point cloud segmentation.
Problem
Semantic point cloud segmentation networks need large amounts of annotated point cloud data, but existing augmentations either keep labels unreliable after manipulation or generate shapes without point-wise labels, so they cannot be used for end-to-end segmentation training.
Approach
A conditional generative network learns the voxelized discrepancy between source and target point clouds in a latent space, samples discrepancy representations to reshape a source cloud into labeled new examples, and mixes them in latent space weighted by segmentation cross-entropy so that hard examples feed back into training both the generator and segmenter.
Key results
- On ShapeNetPart, GHEA with PointMetaBase reaches 86.1 Cat. mIoU / 87.8 Ins. mIoU versus 84.2 / 86.4 for the backbone w/o augmentation... (PointMetaBase baseline is 84.2/86.4 in Table 1), improving 0.8-2.1 Cat. mIoU across four backbones.
- On ScanNet V2 test, GHEA with DeLA reaches 75.0 test mIoU and PointNeXt reaches 73.4, versus 72.8 and 67.4 without augmentation.
- On S3DIS Area 5, GHEA with DeLA reaches 74.9 mIoU versus 73.0 without GHEA.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add GHEA to DeLA | DeLA · ShapeNetPart test | Cat. mIoU: 85.8 -> 86.6 | Helped |
| Add GHEA to DeLA | DeLA · S3DIS Area5 | mIoU: 73.0 -> 74.9 | Helped |
| Add GHEA to DeLA | DeLA · S3DIS 6-fold | mIoU: 76.9 -> 79.5 | Helped |
| Add GHEA to DeLA | DeLA · ScanNet V2 | Val. mIoU: 73.8 -> 76.6 | Helped |
| Add GHEA to DeLA | DeLA · ScanNet V2 | Test mIoU: 72.8 -> 75.0 | Helped |
| Replace generative point-cloud reshape with random manipulation, discriminative reshaping, or part-label-conditioned generation (ablation) | GHEA with PointMetaBase · ShapeNetPart test | Cat. mIoU: 86.1 -> 85.4; 85.1; 82.0; 84.7; 83.8 | Hurt |
| Replace generative point-cloud reshape with alternative methods (ablation) | GHEA with PointMetaBase · ShapeNetPart test | Ins. mIoU: 87.8 -> 87.2; 87.1; 84.1; 87.0; 86.2 | Hurt |
| Remove example mixup and choose one generated example, or replace error-based weights with same or random weights (ablation) | GHEA with PointMetaBase · ShapeNetPart test | Cat. mIoU: 86.1 -> 85.3; 85.3; 85.1; 85.4; 85.2 | Hurt |
| Remove example mixup and choose one generated example, or replace error-based weights with same or random weights (ablation) | GHEA with PointMetaBase · ShapeNetPart test | Ins. mIoU: 87.8 -> 87.3; 87.2; 87.1; 87.3; 87.2 | Hurt |
| Use hard examples to update both the generative and segmentation networks (ablation) | GHEA with PointMetaBase · ShapeNetPart test | Cat. mIoU: 85.1; 85.1; 85.6 -> 86.1 | Helped |
| Use hard examples to update both the generative and segmentation networks (ablation) | GHEA with PointMetaBase · ShapeNetPart test | Ins. mIoU: 87.1; 87.1; 87.5 -> 87.8 | Helped |
| Add hard examples only as target examples rather than source examples or both (ablation) | GHEA with PointMetaBase · ShapeNetPart test | Cat. mIoU: 85.5; 85.2 -> 86.1 | Helped |
Surprising findings
- PointWOLF random deformation degrades ScanNet V2 and S3DIS results for several backbones, e.g. PointNet++ drops to 40.8 val mIoU versus 50.8 baseline.
- Adding hard examples to the source side of generative training hurts performance compared with adding them only to the target side.
Limitations
- Object-level scene augmentation is more complex and gains rely on the chosen backbones, with reshaping of full scenes described as highly complex.
- Generator trained insufficiently in early epochs yields unreasonable shapes and diversity fluctuations.
- Authors state supplementary file holds critical implementation details; the main text alone may be insufficient for reproduction.
Experimental setup
Datasets: ShapeNetPart S3DIS ScanNet V2
Benchmarks: ShapeNetPart S3DIS ScanNet V2
Metrics: Category mIoU Instance mIoU mIoU OA mAcc Validation mIoU Test mIoU
Baselines: PointWOLF conventional data augmentation (CDA) PointNet++ PointNeXt PointMetaBase DeLA
Builds on: PointWOLF PointMetaBase PointNeXt DeLA PointNet++
Closest papers by meaning
- Weakly Supervised Semantic Segmentation for Large-Scale Point Cloud (AAAI 2021 · similarity 0.86)
- Generated and Pseudo Content guided Prototype Refinement for Few-shot Point Cloud Segmentation (NeurIPS 2024 · similarity 0.86)
- Generalized Few-shot 3D Point Cloud Segmentation with Vision-Language Model (CVPR 2025 · similarity 0.85)
- Point Cloud Semantic Segmentation with Sparse and Inhomogeneous Annotations (AAAI 2025 · similarity 0.84)
- Construct to Associate: Cooperative Context Learning for Domain Adaptive Point Cloud Segmentation (CVPR 2024 · similarity 0.84)
- FreePoint: Unsupervised Point Cloud Instance Segmentation (CVPR 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.