Research map / 3D vision / 3D generation
SINGAPO: Single Image Controlled Generation of Articulated Parts in Objects
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Generates articulated household objects from a single resting-state image via GPT-4o part-graph prediction and a CAGE-based diffusion transformer with DINOv2 image cross-attention, followed by part mesh retrieval.
Problem
Creating 3D articulated object assets from a single RGB image of an object in its resting (closed) state, where occlusion and small parts make part shape and articulation ambiguous.
Approach
A coarse-to-fine pipeline: a VLM (GPT-4o) predicts a part connectivity graph from the image, a diffusion transformer (built on CAGE) generates per-part abstract attributes conditioned on DINOv2 patch features via image cross-attention and graph attention masks, and part meshes are retrieved from a library to assemble the object. Classifier-free training over category, graph, and image enables gene
Key results
- On PartNet-Mobility (single output), Ours achieves RS-d_CD 0.0168 and graph accuracy 82.47% vs NAP-ICA 0.0173 / 75.97% and URDFormer 0.5502 / 32.03%.
- On zero-shot ACD, Ours achieves RS-d_gIoU 0.9517 vs NAP-ICA 1.0233 and URDFormer 1.2288.
- In a 48-participant user study, the method ranked first with average rank 1.20 vs NAP-ICA 1.91 and URDFormer 2.89.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use SINGAPO rather than NAP-ICA | articulated object generation · PartNet-Mobility test set | RS-d_gIoU: 0.5778 -> 0.4841 | Helped |
| Use SINGAPO rather than NAP-ICA | articulated object generation · ACD test set | RS-d_gIoU: 1.0233 -> 0.9517 | Helped |
| Use SINGAPO rather than NAP-ICA | articulated object generation · PartNet-Mobility test set | Graph accuracy: 75.97 -> 82.47 | Helped |
| Add image cross-attention (ablation) | CAGE base model · ACD test set | RS-d_CD: 0.3469 -> 0.1459 | Helped |
| Remove category as a hard constraint using classifier-free training (ablation) | Base + ICA · ACD test set | RS-d_CD: 0.1459 -> 0.1423 | Helped |
| Add foreground loss (ablation) | Base + ICA + cCF · ACD test set | RS-d_CD: 0.1423 -> 0.1403 | Helped |
| Add classifier-free training for image guidance (ablation) | Base + ICA + cCF + L_fg · ACD test set | RS-d_CD: 0.1403 -> 0.1184 | Helped |
| Add classifier-free training for graph constraint (ablation) | Base + ICA + cCF + L_fg + iCF · ACD test set | RS-d_CD: 0.1184 -> 0.1014 | Helped |
| Use augmented training data (ablation) | SINGAPO · PartNet-Mobility | RS-d_gIoU: 0.5965 -> 0.4414 | Helped |
| Use augmented training data (ablation) | SINGAPO · ACD | RS-d_gIoU: 1.1270 -> 0.9528 | Helped |
Surprising findings
- The original CAGE conditioning on object category and graph as hard constraints overfit and generalized poorly; classifier-free dropping of these conditions improved generalization.
- Reported ID and AID metrics sometimes give lower scores to worse reconstructions, motivating the authors' Hungarian-matched metrics.
Limitations
- Tailored to cuboid cabinet-like categories; mesh retrieval from a library fails to capture fine geometric details in the input image.
- Failures on low-resolution inputs, complex textures, cluttered arrangements, and small parts; graph predictions from GPT-4o can be inaccurate.
- Struggles with images of objects in extreme articulated states, since training used only resting-state images.
Experimental setup
Datasets: PartNet-Mobility ACD
Benchmarks: PartNet-Mobility test set ACD test set
Metrics: RS-d_gIoU AS-d_gIoU RS-d_cDist AS-d_cDist RS-d_CD AS-d_CD AOR graph accuracy ID AID
Baselines: URDFormer NAP-ICA Real2Code
Builds on: CAGE NAP DINOv2 denoising diffusion probabilistic model
Closest papers by meaning
- DIPO: Dual-State Images Controlled Articulated Object Generation Powered by Diverse Data (NeurIPS 2025 · similarity 0.92)
- CAGE: Controllable Articulation GEneration (CVPR 2024 · similarity 0.91)
- ArtFormer: Controllable Generation of Diverse 3D Articulated Objects (CVPR 2025 · similarity 0.90)
- MeshArt: Generating Articulated Meshes with Structure-Guided Transformers (CVPR 2025 · similarity 0.90)
- General Articulated Objects Manipulation in Real Images via Part-Aware Diffusion Process (NeurIPS 2024 · similarity 0.89)
- Articulate your NeRF: Unsupervised articulated object modeling via conditional view synthesis (NeurIPS 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.