Research map / Vision-language and multimodal / Human and embodied modeling
PartNeXt: A Next-Generation Dataset for Fine-Grained and Hierarchical 3D Part Understanding
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces PartNeXt, a 23.5K-model textured 3D part dataset across 50 categories with hierarchical part labels from a web-based face-selection annotation tool, and benchmarks segmentation, part-centric 3D QA, and Point-SAM training.
Problem
Existing 3D part datasets such as PartNet use untextured, remeshed geometry and require expert annotation, limiting scale, texture-aware labels, and crowdsourcing; current part segmentation and 3D LLMs struggle with fine-grained part understanding.
Approach
Annotate parts directly on original textured meshes via a crowdsourced web interface with dual-panel layout and face-level selection tools, using CLIP-based category filtering and GPT-4o to bootstrap part hierarchies that experts then refine.
Key results
- On PartNeXt segmentation, SAMesh reaches 51.57 mIoU and PartField 50.22 mIoU, while SAMPart3D reaches 36.78 mIoU across 10 reported categories.
- 3D LLMs perform poorly on part-centric QA: best part-count MAE is 1.72 (ShapeLLM, with category) and grounding IoU is only 0.33 (ShapeLLM).
- Point-SAM trained on PartNeXt+PartNet mixture reaches 40.4 IoU@1 on PartNet-Mobility vs 39.0 for PartNet alone, and 45.3 vs 39.9 IoU@1 on PartNeXt test.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Train Point-SAM on PartNeXt instead of PartNet | Point-SAM · PartNet-Mobility | IoU@10: 62.9 -> 67.4 | Helped |
| Train Point-SAM on PartNeXt instead of PartNet | Point-SAM · PartNeXt | IoU@10: 60.3 -> 65.9 | Helped |
| Train Point-SAM on PartNet and PartNeXt instead of PartNet alone | Point-SAM · PartNet-Mobility | IoU@10: 62.9 -> 68.7 | Helped |
| Train Point-SAM on PartNet and PartNeXt instead of PartNet alone | Point-SAM · PartNeXt | IoU@10: 60.3 -> 67.6 | Helped |
| Provide the object category in the prompt | 3DLLM · PartNeXt part counting | MAE: 2.46 -> 2.16 | Helped |
| Provide the object category in the prompt | PointLLM · PartNeXt part counting | MAE: 1.79 -> 1.87 | Hurt |
| Provide the object category in the prompt | ShapeLLM · PartNeXt part counting | MAE: 1.85 -> 1.72 | Helped |
| Provide the object category in the prompt | PointLLM · PartNeXt part classification | accuracy: 0.18 -> 0.22 | Helped |
| Provide the object category in the prompt | ShapeLLM · PartNeXt part classification | accuracy: 0.08 -> 0.25 | Helped |
| Provide the object category in the prompt | ShapeLLM · PartNeXt part grounding | IoU: 0.30 -> 0.33 | Helped |
Surprising findings
- Point-SAM trained only on PartNeXt (40.2 IoU@1) outperforms PartNet-trained Point-SAM on PartNet-Mobility at most prompt counts, despite PartNeXt's annotations being produced by non-experts.
- Sorted by category, PartField is weaker than SAMesh on bed and chair, showing no single method dominates across categories.
Limitations
- Only 23,519 models; expansion via Objaverse-XL is ongoing.
- Each category needs a predefined fine-grained hierarchy, limiting open-vocabulary annotation.
- Only plain part names are provided; no captions or physical attributes.
Experimental setup
Datasets: Objaverse ABO 3D-FUTURE PartNet PartNet-Mobility Cap3D
Benchmarks: PartNeXt class-agnostic segmentation benchmark PartNeXt part-centric 3D QA PartNet-Mobility
Metrics: mIoU MAE accuracy IoU@k
Baselines: PartField SAMPart3D SAMesh PointLLM ShapeLLM 3D-LLM Point-SAM
Builds on: PartNet Point-SAM Objaverse Cap3D
Closest papers by meaning
- 3DCoMPaT200: Language Grounded Large-Scale 3D Vision Dataset for Compositional Recognition (NeurIPS 2024 · similarity 0.92)
- PartDistill: 3D Shape Part Segmentation by Vision-Language Model Distillation (CVPR 2024 · similarity 0.89)
- PartGen: Part-level 3D Generation and Reconstruction with Multi-view Diffusion Models (CVPR 2025 · similarity 0.89)
- PARTONOMY: Large Multimodal Models with Part-Level Visual Understanding (NeurIPS 2025 · similarity 0.89)
- PartCrafter: Structured 3D Mesh Generation via Compositional Latent Diffusion Transformers (NeurIPS 2025 · similarity 0.89)
- Generative 3D Part Assembly via Part-Whole-Hierarchy Message Passing (CVPR 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.