Research map / Video / Video generation
DIPO: Dual-State Images Controlled Articulated Object Generation Powered by Diverse Data
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes DIPO, a diffusion-based generator of articulated 3D objects conditioned on resting and articulated image pairs with a CoT graph reasoner, plus the LEGO-Art pipeline and PM-X dataset of 600 complex objects.
Problem
Controllable generation of articulated 3D objects (part layout, joint axes, motion ranges, connectivity) from minimal input, where single-image or category-prior methods lack motion cues and existing datasets lack structural complexity.
Approach
Condition a diffusion transformer on a pair of resting and articulated state images via a Dual-State Injection Module, use a chain-of-thought Graph Reasoner to infer part connectivity as an attention mask, and expand data with an LLM/VLM agent pipeline (LEGO-Art) that assembles PartNet-Mobility parts into more complex objects.
Key results
- On the PartNet-Mobility test set, DIPO reaches 85.06% graph accuracy versus 75.97% for SINGAPO and reduces RS gIoU from 0.5134 to 0.4561.
- On the ACD out-of-distribution test set, DIPO reaches 48.15% graph accuracy versus 36.67% for SINGAPO.
- In ablation on ACD, the full model (PM-X + DIM + GR) gives the best reconstruction, with RS gIoU 0.9126 versus 0.9872 without any component.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use DIPO instead of SINGAPO | Articulated object generation · PartNet-Mobility test | AS-d_CD: 0.1270 -> 0.0423 | Helped |
| Use DIPO instead of SINGAPO | Articulated object generation · ACD test | Graph accuracy (%): 36.67 -> 48.15 | Helped |
| Add PM-X data (ablation) | DIPO · ACD test | RS-d_gIoU: 0.9872 -> 0.9429 | Helped |
| Add DIM (ablation) | DIPO · ACD test | RS-d_gIoU: 0.9872 -> 0.9565 | Helped |
| Add GR without PM-X or DIM (ablation) | DIPO · ACD test | RS-d_gIoU: 0.9872 -> 0.9902 | Hurt |
| Add PM-X and DIM (ablation) | DIPO · ACD test | RS-d_gIoU: 0.9872 -> 0.9212 | Helped |
| Add PM-X and GR (ablation) | DIPO · ACD test | RS-d_gIoU: 0.9872 -> 0.9332 | Helped |
| Add DIM and GR without PM-X (ablation) | DIPO · ACD test | RS-d_gIoU: 0.9872 -> 0.9497 | Helped |
| Add PM-X, DIM, and GR (ablation) | DIPO · ACD test | RS-d_gIoU: 0.9872 -> 0.9126 | Helped |
| Remove CoT from GR (ablation) | DIPO Graph Reasoner · ACD test | Graph accuracy (%): 39.26 vs 48.15 | Hurt |
| Remove visual input from GR (ablation) | DIPO Graph Reasoner · ACD test | Graph accuracy (%): 37.77 vs 48.15 | Hurt |
| Remove dual-state input from GR (ablation) | DIPO Graph Reasoner · ACD test | Graph accuracy (%): 39.63 vs 48.15 | Hurt |
Surprising findings
- The Graph Reasoner does not consistently help; without PM-X training it can produce more complex topologies that are out-of-distribution and degrade performance.
Limitations
- Only StorageFurniture and Table categories are included in PM-X and evaluated.
- Part geometry is retrieved from existing meshes rather than generated, so geometric precision is limited.
- Evaluation on real-world data is qualitative only.
Experimental setup
Datasets: PartNet-Mobility PM-X ACD
Benchmarks: PartNet-Mobility test split ACD test set
Metrics: gIoU center distance Chamfer distance graph accuracy
Baselines: URDFormer NAP-ICA SINGAPO
Builds on: SINGAPO CAGE NAP DINOv2 chain-of-thought prompting
Closest papers by meaning
- SINGAPO: Single Image Controlled Generation of Articulated Parts in Objects (ICLR 2025 · similarity 0.92)
- CAGE: Controllable Articulation GEneration (CVPR 2024 · similarity 0.89)
- ArtFormer: Controllable Generation of Diverse 3D Articulated Objects (CVPR 2025 · similarity 0.88)
- MeshArt: Generating Articulated Meshes with Structure-Guided Transformers (CVPR 2025 · similarity 0.88)
- ArticulatedGS: Self-supervised Digital Twin Modeling of Articulated Objects using 3D Gaussian Splatting (CVPR 2025 · similarity 0.87)
- REACTO: Reconstructing Articulated Objects from a Single Video (CVPR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.