Research map / Image generation / Autoregressive and latent generation
MIDGArD: Modular Interpretable Diffusion over Graphs for Articulated Designs
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces MIDGArD, a two-stage diffusion framework that generates interpretable articulation graphs (structure generator) and then populates links with bounding-box-constrained 3D meshes via an adapted SDFusion (shape generator), producing simulatable articulated assets.
Problem
Generating articulated 3D objects that are geometrically detailed, kinematically plausible, consistent, controllable and interpretable remains hard; prior end-to-end methods (NAP, CAGE) produce inconsistent shapes and opaque latents.
Approach
A modular pipeline: a GAT-based graph diffusion over categorical, image-latent and Plücker-joint node/edge features, with diffusion directly on the Plücker manifold via a spherical parametrization, followed by a bounding-box-constrained SDFusion shape generator conditioned on the denoised graph, plus PCA/OBB-based canonical orientation of parts.
Key results
- Unconditional generation: MMD 0.0264 vs NAP 0.0282 (6.4% improvement) and COV 0.4857 vs 0.4675 (3.7% improvement); 1-NNA tied at 0.5831.
- Joint-type distribution of generated objects (2% screw, 62% revolute, 36% prismatic) closely matches training data, vs NAP's 95% screw; chi-square 12.7 vs 5618.
- Bounding-box-constrained shape generation with bb prior + bb MLP reaches Chamfer distance 0.0072 (generated) vs 0.0307 for original SDFusion + bb MLP; OBB preprocessing reduces bounding-box volume by 17.4% on average.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add Plücker manifold parameterization (ablation) | MIDGArD · PartNet Mobility, unconditional generation | MMD: 0.0270 -> 0.0264 | Helped |
| add Plücker manifold parameterization (ablation) | MIDGArD · PartNet Mobility, unconditional generation | COV: 0.4779 -> 0.4857 | Helped |
| add Plücker manifold parameterization (ablation) | MIDGArD · PartNet Mobility, unconditional generation | 1-NNA: 0.6221 -> 0.5831 | Helped |
| use MIDGArD with Plücker manifold rather than NAP | PartNet Mobility unconditional generation · PartNet Mobility | MMD: 0.0282 -> 0.0264 | Helped |
| use MIDGArD with Plücker manifold rather than NAP | PartNet Mobility unconditional generation · PartNet Mobility | COV: 0.4675 -> 0.4857 | Helped |
| use MIDGArD with Plücker manifold rather than NAP | PartNet Mobility unconditional generation · PartNet Mobility | 1-NNA: 0.5831 -> 0.5831 | No clear effect |
| add the bounding-box prior to shape generation (ablation) | SDFusion with bb MLP and oriented dataset · PartNet Mobility, shape generation | CD (generated): 0.0152 -> 0.0072 | Helped |
| add the bounding-box prior to shape generation (ablation) | SDFusion with bb MLP and oriented dataset · PartNet Mobility, shape generation | OBBcount: 1490 -> 1146 | Helped |
| add the bounding-box prior to shape generation (ablation) | SDFusion with bb MLP and oriented dataset · PartNet Mobility, shape generation | OBBsum: 37.15 -> 10.89 | Helped |
| preprocess the dataset into oriented canonical poses (ablation) | SDFusion shape generation · PartNet Mobility, shape generation | CD (generated): 0.0307 -> 0.0152 | Helped |
| replace the bb MLP embedding with a tree GAT (ablation) | SDFusion with bounding-box prior · PartNet Mobility, shape generation | CD (generated): 0.0072 -> 0.0076 | Hurt |
| replace the bb MLP embedding with a graph GAT (ablation) | SDFusion with bounding-box prior · PartNet Mobility, shape generation | CD (generated): 0.0072 -> 0.0084 | Hurt |
Surprising findings
- Orientation-aware OBB preprocessing and bounding-box prior substantially improve results, while GAT encoders for part conditioning underperform a simple MLP embedding.
Limitations
- Instantiation Distance is point-cloud based and misses unrealistic motion of small parts; benchmarks for articulated generation are limited.
- Graph size is capped at N=8 nodes, limiting scalability to more complex objects.
- Shape generation occasionally yields geometry or kinematic issues, and 10 categories were excluded from shape training.
Experimental setup
Datasets: PartNet-Mobility
Benchmarks: PartNet-Mobility
Metrics: Instantiation Distance (ID) minimum matching distance (MMD) coverage (COV) 1-nearest-neighbor accuracy (1-NNA) chi-square statistic on joint type distribution OBBcount OBBsum Chamfer distance
Baselines: NAP SDFusion (original)
Builds on: NAP CAGE SDFusion DDIM DDPM
Closest papers by meaning
- CAGE: Controllable Articulation GEneration (CVPR 2024 · similarity 0.89)
- MeshArt: Generating Articulated Meshes with Structure-Guided Transformers (CVPR 2025 · similarity 0.88)
- PartGen: Part-level 3D Generation and Reconstruction with Multi-view Diffusion Models (CVPR 2025 · similarity 0.87)
- CraftsMan3D: High-fidelity Mesh Generation with 3D Native Diffusion and Interactive Geometry Refiner (CVPR 2025 · similarity 0.87)
- Sculpt3D: Multi-View Consistent Text-to-3D Generation with Sparse 3D Prior (CVPR 2024 · similarity 0.87)
- SINGAPO: Single Image Controlled Generation of Articulated Parts in Objects (ICLR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.