Research map / 3D vision / 3D generation
AutoPartGen: Autoregressive 3D Part Generation and Discovery
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes AutoPartGen, an autoregressive latent-diffusion (DiT over 3DShape2VecSet latents) model that generates 3D objects part by part from an image, 2D part masks, or an existing 3D shape.
Problem
Generating compositional 3D objects whose parts fit together coherently, with an unknown and variable number of parts, from images, 2D masks, or existing 3D shapes, without part annotations or extra optimization.
Approach
Observes that 3DShape2VecSet latents are compositional (concatenated codes decode to the union of surfaces), and uses a single conditional latent diffusion model that autoregressively generates one part latent at a time conditioned on previously generated parts, the whole-object code, and optional image/mask evidence, stopping with an EoT token.
Key results
- On PartObjaverse-Tiny part completion, AutoPartGen reaches 0.665 part IoU and 0.047 part CD, versus 0.658/0.065 for HoloPart and 0.614/0.121 for PartGen.
- Whole-object IoU is 0.832 for AutoPartGen versus 0.821 for HoloPart and 0.779 for PartGen.
- Ablation shows autoregressive conditioning raises part IoU from 0.574 to 0.633 (200-epoch models).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| enable autoregressive conditioning (ablation) | AutoPartGen · Part completion | IoU: 0.574 -> 0.633 | Helped |
| enable autoregressive conditioning (ablation) | AutoPartGen · Part completion | F-Score: 0.795 -> 0.825 | Helped |
| enable autoregressive conditioning (ablation) | AutoPartGen · Part completion | CD: 0.067 -> 0.052 | Helped |
| enable autoregressive conditioning (ablation) | AutoPartGen · Overall shape | IoU: 0.783 -> 0.811 | Helped |
| enable autoregressive conditioning (ablation) | AutoPartGen · Overall shape | F-Score: 0.917 -> 0.934 | Helped |
| enable autoregressive conditioning (ablation) | AutoPartGen · Overall shape | CD: 0.031 -> 0.022 | Helped |
| use geometry guidance 4 and image guidance 7 (ablation) | AutoPartGen · Part completion | IoU: 0.665 | Helped |
| use geometry guidance 4 and image guidance 7 (ablation) | AutoPartGen · Part completion | F-Score: 0.861 | Helped |
| use geometry guidance 4 and image guidance 7 (ablation) | AutoPartGen · Part completion | CD: 0.047 | Helped |
Surprising findings
- HoloPart, which receives the 3D partial object and 3D masks, outperforms PartGen in the authors' comparison, which they attribute to richer input information.
Limitations
- Can only generate bounded scenes, inherited from the VAE latent space.
- Lacks explicit granularity control outside the masks-to-parts setting.
- Learned part distributions may inherit dataset biases; identical parts are sometimes generated together, a failure mode.
Experimental setup
Datasets: PartObjaverse-Tiny Google Scanned Objects
Benchmarks: PartObjaverse-Tiny Google Scanned Objects
Metrics: IoU F-score Chamfer Distance
Baselines: PartGen HoloPart
Builds on: 3DShape2VecSet PartGen DiT DINOv2
Closest papers by meaning
- PartGen: Part-level 3D Generation and Reconstruction with Multi-view Diffusion Models (CVPR 2025 · similarity 0.92)
- PartCrafter: Structured 3D Mesh Generation via Compositional Latent Diffusion Transformers (NeurIPS 2025 · similarity 0.91)
- SAR3D: Autoregressive 3D Object Generation and Understanding via Multi-scale 3D VQVAE (CVPR 2025 · similarity 0.89)
- ARMesh: Autoregressive Mesh Generation via Next-Level-of-Detail Prediction (NeurIPS 2025 · similarity 0.88)
- Learning Partonomic 3D Reconstruction from Image Collections (CVPR 2025 · similarity 0.88)
- Generative 3D Part Assembly via Part-Whole-Hierarchy Message Passing (CVPR 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.