Research map / 3D vision / Face and shape
Learning Partonomic 3D Reconstruction from Image Collections
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Learns partonomic (part-decomposed) 3D shape reconstruction from single-view images using only 2D object and part masks, via a compositional neural-field representation and a part transformer.
Problem
Single-view 3D reconstruction from image collections with only 2D annotations typically yields whole-object shapes and ignores semantic part structure; frequent part occlusion and a larger solution space make direct learning hard.
Approach
Represents shape as a hierarchical composition of a conditional shape template (coarse shape and soft part assignment) and per-part deformation fields, and infers their latents with a part transformer whose learnable part tokens cross-attend to image features and self-attend to model part-part relations, trained end-to-end with image, object-mask, and part-mask 2D losses under differentiable rende
Key results
- On ShapeNetPart, the method reaches average Chamfer-L1 0.197 vs 0.217 for AST* and 0.249 for Unicorn*, and part mIoU 0.514 vs 0.446 for AST*.
- On PartNet, average Part Chamfer-L1 is 0.403 vs 0.414 for AST* and 0.480 for Unicorn*, and Chamfer-L1 is 0.209 vs 0.223/0.224 for the baselines.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use the proposed compositional representation and part transformer instead of AST* | Partonomic reconstruction · ShapeNetPart | Chamfer-L1: 0.217 -> 0.197 | Helped |
| Use the proposed compositional representation and part transformer instead of Unicorn* | Partonomic reconstruction · ShapeNetPart | Part Chamfer-L1: 0.446 -> 0.379 | Helped |
| Use the proposed compositional representation and part transformer instead of AST* | Partonomic reconstruction · ShapeNetPart | Part Classification Accuracy: 0.610 -> 0.675 | Helped |
| Use the proposed compositional representation and part transformer instead of Unicorn* | Partonomic reconstruction · ShapeNetPart | Part mIoU: 0.458 -> 0.514 | Helped |
| Use the proposed compositional representation and part transformer instead of Unicorn* | Partonomic reconstruction · PartNet | Chamfer-L1: 0.223 -> 0.209 | Helped |
| Use the proposed compositional representation and part transformer instead of AST* | Partonomic reconstruction · PartNet | Part Chamfer-L1: 0.414 -> 0.403 | Helped |
| Add PDFs to the CST base model, with MLP-predicted object and part latent vectors (ablation) | Partonomic reconstruction · ShapeNetPart | Chamfer-L1: 0.239 -> 0.228 | Helped |
| Add PDFs to the CST base model, with MLP-predicted object and part latent vectors (ablation) | Partonomic reconstruction · ShapeNetPart | Part Chamfer-L1: 0.667 -> 0.633 | Helped |
| Add the part transformer to the CST and PDF model (ablation) | Partonomic reconstruction · ShapeNetPart | Chamfer-L1: 0.228 -> 0.197 | Helped |
| Add the part transformer to the CST and PDF model (ablation) | Partonomic reconstruction · ShapeNetPart | Part Chamfer-L1: 0.633 -> 0.379 | Helped |
| Add PDFs to the CST base model, with MLP-predicted object and part latent vectors (ablation) | Partonomic reconstruction · PartNet | Chamfer-L1: 0.241 -> 0.229 | Helped |
| Add PDFs to the CST base model, with MLP-predicted object and part latent vectors (ablation) | Partonomic reconstruction · PartNet | Part Chamfer-L1: 0.518 -> 0.498 | Helped |
Limitations
- Baselines are self-constructed extensions of Unicorn and AST since no prior work addresses this task.
- Evaluated only on five categories per dataset and trained per category.
- Qualitative only on CUB-200-2011 with no quantitative part metrics.
Experimental setup
Datasets: ShapeNetPart PartNet CUB-200-2011
Benchmarks: ShapeNetPart PartNet CUB-200-2011
Metrics: Chamfer-L1 Part Chamfer-L1 Part Classification Accuracy Part mIoU
Baselines: Unicorn* AST* Base Model +Deform
Builds on: Unicorn AST U-CMR CMR
Closest papers by meaning
- VLS: A Vision-Language-Shape Model for Open-Vocabulary Partonomic 3D Reconstruction (NeurIPS 2026 · similarity 0.92)
- PartCrafter: Structured 3D Mesh Generation via Compositional Latent Diffusion Transformers (NeurIPS 2025 · similarity 0.90)
- 3DCoMPaT200: Language Grounded Large-Scale 3D Vision Dataset for Compositional Recognition (NeurIPS 2024 · similarity 0.89)
- PartGen: Part-level 3D Generation and Reconstruction with Multi-view Diffusion Models (CVPR 2025 · similarity 0.89)
- AutoPartGen: Autoregressive 3D Part Generation and Discovery (NeurIPS 2025 · similarity 0.88)
- PartNeXt: A Next-Generation Dataset for Fine-Grained and Hierarchical 3D Part Understanding (NeurIPS 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.