Research map / Detection and segmentation / 3D detection and driving
ConceptFactory: Facilitate 3D Object Knowledge Annotation with Object Conceptualization
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces ConceptFactory, a web-based object conceptualization toolbox (STL-C with 263 concept templates) and a 4380-object conceptualized asset, so 3D object knowledge (segmentation, part pose, affordance) can be procedurally annotated once per object.
Problem
Annotating 3D object knowledge (semantics, pose, affordance) is costly because each knowledge type is labeled separately per object, and affordance labels are hard to collect manually.
Approach
Describe each object with generalized geometric concept templates (inspired by Recognition-by-Components), fit them with a differentiable parameter optimizer, then procedurally define knowledge on templates and propagate it to objects via point-wise correspondence.
Key results
- Semantic segmentation with ConceptFactory annotations matches original labels closely (PointTransformer mIoU 75.6 vs 75.4 original).
- Affordance annotations improve Where2Act push success rate from 23.7 to 30.0 (up to 26.6% relative gain on push).
- Conceptualization takes about 7 min per object versus about 10 min for conventional part pose annotation; ConceptFactory Asset has 4380 objects in 39 categories.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Train PointTransformer with ConceptFactory annotations instead of original annotations | PointTransformer · Semantic segmentation | mAcc: 90.0 -> 89.8 | Hurt |
| Train PointTransformer with ConceptFactory annotations instead of original annotations | PointTransformer · Semantic segmentation | mIoU: 75.4 -> 75.6 | Helped |
| Train PointNet++ with ConceptFactory annotations instead of original annotations | PointNet++ · Semantic segmentation | mAcc: 88.8 -> 88.8 | No clear effect |
| Train PointNet++ with ConceptFactory annotations instead of original annotations | PointNet++ · Semantic segmentation | mIoU: 67.2 -> 67.3 | Helped |
| Train GAPartNet with ConceptFactory annotations instead of original annotations | GAPartNet · Cross-category segmentation | mAP: 30.0 -> 30.6 | Helped |
| Train GAPartNet with ConceptFactory annotations instead of original annotations | GAPartNet · Cross-category segmentation | mAP50: 37.7 -> 38.5 | Helped |
| Train GAPartNet with ConceptFactory annotations instead of original annotations | GAPartNet · Part pose estimation | mIoU: 42.6 -> 42.8 | Helped |
| Train GAPartNet with ConceptFactory annotations instead of original annotations | GAPartNet · Part pose estimation | A5: 29.1 -> 30.3 | Helped |
| Train GAPartNet with ConceptFactory annotations instead of original annotations | GAPartNet · Part pose estimation | A10: 55.7 -> 56.9 | Helped |
| Train Where2Act with ConceptFactory affordance annotations instead of original annotations | Where2Act · Manipulation tasks (push) | Success rate: 23.7 -> 30.0 | Helped |
| Train Where2Act with ConceptFactory affordance annotations instead of original annotations | Where2Act · Manipulation tasks (pull) | Success rate: 8.4 -> 10.6 | Helped |
| Train Where2Explore with ConceptFactory affordance annotations instead of original annotations | Where2Explore · Manipulation tasks (push) | Success rate: 33.0 -> 37.4 | Helped |
Surprising findings
- Using ConceptFactory affordance labels improves Where2Act and Where2Explore over simulation-acquired affordance labels, which the authors attribute to less noise and avoiding simulation-induced label errors.
Limitations
- Object conceptualization still requires human effort per object (about 7 min).
- Vision annotations are evaluated against original human labels, so slight performance drops may reflect annotator variance.
Experimental setup
Datasets: ConceptFactory Asset ShapeNet-derived objects (original annotations) PartNet-Mobility
Benchmarks: ShapeNetPart semantic segmentation (PointTransformer, PointNet++) GAPartNet cross-category segmentation GAPartNet part pose estimation Where2Act manipulation Where2Explore manipulation SAPIEN simulator manipulation tasks
Metrics: mAcc mIoU mAP mAP50 A5 A10 success rate
Baselines: PointTransformer PointNet++ GAPartNet Where2Act Where2Explore
Builds on: Recognition-by-Components BPS Point2Mesh loss GAPartNet Where2Act
Closest papers by meaning
- Discovering Conceptual Knowledge with Analytic Ontology Templates for Articulated Objects (AAAI 2025 · similarity 0.85)
- ImageNet3D: Towards General-Purpose Object-Level 3D Understanding (NeurIPS 2024 · similarity 0.85)
- 3DCoMPaT200: Language Grounded Large-Scale 3D Vision Dataset for Compositional Recognition (NeurIPS 2024 · similarity 0.84)
- Imagine: Image-Guided 3D Part Assembly with Structure Knowledge Graph (AAAI 2025 · similarity 0.84)
- RoboSpatial: Teaching Spatial Understanding to 2D and 3D Vision-Language Models for Robotics (CVPR 2025 · similarity 0.84)
- Surprise3D: A Dataset for Spatial Understanding and Reasoning in Complex 3D Scenes (NeurIPS 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.