Research map / Detection and segmentation / Open-set and few-shot detection
3DCoMPaT200: Language Grounded Large-Scale 3D Vision Dataset for Compositional Recognition
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces 3DCoMPaT200, a 19k-shape, 200-category part- and material-annotated 3D dataset, and benchmarks point-cloud models on classification, segmentation, grounded compositional recognition, and a new text-to-shape compositional retrieval task using ULIP-style models.
Problem
Existing part-level 3D datasets cover few object categories and lack material annotations, limiting compositional part-material understanding of 3D shapes.
Approach
Extends 3DCoMPaT to 200 object categories with fine- and coarse-grained part labels and 293 fine-grained material classes, and proposes compositional part shape retrieval where captions describe one, three, or six parts with GPT-4-generated color descriptions.
Key results
- PointTransformer V3 reaches 88.48% shape accuracy and 88.48 instance accuracy in classification, the best among tested models in the paper's table.
- Uni3D achieves the best part-based retrieval at 59.4% R@1 for 6-part captions, versus 26.74% R@1 for 1-part captions.
- PointTransformer V3 obtains 42.80% Grounded-value-all (fine-grained) and 72.91% (coarse-grained) in grounded compositional recognition.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add a shape-class embedding prior to PCT (ablation) | PCT part segmentation · 3DCoMPaT200, fine-grained part segmentation, 10 compositions | mIoU (%): 48.70 -> 81.83 | Helped |
| Add a shape-class embedding prior to PCT (ablation) | PCT part segmentation · 3DCoMPaT200, coarse-grained part segmentation, 10 compositions | mIoU (%): 76.61 -> 91.83 | Helped |
| Increase caption composition from 1 part to 3 parts (ablation) | ULIP · Compositional shape retrieval | R@1 (%): 23.04 -> 37.41 | Helped |
| Increase caption composition from 3 parts to 6 parts (ablation) | ULIP · Compositional shape retrieval | R@1 (%): 37.41 -> 54.8 | Helped |
| Increase caption composition from 1 part to 6 parts (ablation) | Uni3D · Compositional shape retrieval | R@1 (%): 26.74 -> 59.4 | Helped |
| Increase caption composition from 1 part to 6 parts (ablation) | OpenShape · Compositional shape retrieval | R@1 (%): 24.53 -> 56.62 | Helped |
Surprising findings
- PCT achieves 81.83% fine-grained part mIoU with a shape prior but only 48.70% without it, showing large dependence on class conditioning.
- PCT has much lower shape accuracy (71.71%) than PointNet++ (88.32%) despite strong segmentation results.
Limitations
- Not all possible categories are covered due to lack of suitable styling and possible duplicate shapes.
- Experiments use only one composition for most results, with no error bars reported.
Experimental setup
Datasets: 3DCoMPaT200
Benchmarks: 3DCoMPaT200
Metrics: instance accuracy class average accuracy mIoU Shape Accuracy Value Value-All Grounded-value Grounded-value-all Top1 Top5
Baselines: PointNet++ PCT CurveNet PointBert PointTransformer V2 PointTransformer V3 ULIP Uni3D OpenShape
Builds on: 3DCoMPaT ULIP OpenShape Uni3D PointNet++
Closest papers by meaning
- PartNeXt: A Next-Generation Dataset for Fine-Grained and Hierarchical 3D Part Understanding (NeurIPS 2025 · similarity 0.92)
- Learning Partonomic 3D Reconstruction from Image Collections (CVPR 2025 · similarity 0.89)
- ImageNet3D: Towards General-Purpose Object-Level 3D Understanding (NeurIPS 2024 · similarity 0.89)
- Imagine: Image-Guided 3D Part Assembly with Structure Knowledge Graph (AAAI 2025 · similarity 0.87)
- Uni3D: Exploring Unified 3D Representation at Scale (ICLR 2024 · similarity 0.87)
- ImOV3D: Learning Open Vocabulary Point Clouds 3D Object Detection from Only 2D Images (NeurIPS 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.