Research map / 3D vision / 3D generation
PartRM: Modeling Part-Level Dynamics with Large Cross-State Reconstruction Model
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes PartRM, a feed-forward 4D reconstruction model built on LGM that predicts multi-view 3D Gaussians of an articulated object's part-level motion from a single image plus drags, trained on the new PartDrag-4D dataset.
Problem
Modeling part-level dynamics of articulated objects from a single image and user drags is slow and 2D-limited in existing video-diffusion approaches (e.g., Puppet-Master), which cannot directly provide 3D representations for simulators.
Approach
Extends a pretrained large Gaussian reconstruction model (LGM) with propagated drag conditions embedded at multiple U-Net scales, and trains in two stages (distillation-style motion learning on splatter images, then photometric appearance learning) to avoid catastrophic forgetting of static 3D priors.
Key results
- PartRM achieves 28.15 PSNR, 0.9531 SSIM and 0.0356 LPIPS on PartDrag-4D, versus 24.91 PSNR for the best baseline (DragAPart, Drag-First).
- PartRM runs in 4.2 s per drag interaction versus 64.9 s to 245.8 s for Puppet-Master with reconstruction time included.
- On Objaverse-Animation-HQ, PartRM reaches 21.38 PSNR versus 19.46 for DiffEditor; ablation shows two-stage training improves PSNR from 22.05 (stage 1 only) and 25.87 (stage 2 only) to 28.15.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace 2D drag-based generation and separate 3D reconstruction with joint 4D reconstruction | PartRM vs. strongest reported baselines · PartDrag-4D | PSNR: 24.91 -> 28.15 | Helped |
| replace 2D drag-based generation and separate 3D reconstruction with joint 4D reconstruction | PartRM vs. strongest reported baselines · Objaverse-Animation-HQ | PSNR: 19.46 -> 21.38 | Helped |
| increase propagated drags from 1 to 10 (ablation) | PartRM · PartDrag-4D | PSNR: 27.06 -> 28.15 | Helped |
| increase propagated drags from 5 to 10 (ablation) | PartRM · PartDrag-4D | PSNR: 27.56 -> 28.15 | Helped |
| add appearance-learning Stage 2 after motion-learning Stage 1 (ablation) | PartRM · PartDrag-4D | PSNR: 22.05 -> 28.15 | Helped |
| add motion-learning Stage 1 before appearance-learning Stage 2 (ablation) | PartRM · PartDrag-4D | PSNR: 25.87 -> 28.15 | Helped |
| replace single 128×128 drag embeddings with multi-scale embeddings (ablation) | PartRM · PartDrag-4D | PSNR: 25.48 -> 28.15 | Helped |
| replace single 32×32 drag embeddings with multi-scale embeddings (ablation) | PartRM · PartDrag-4D | PSNR: 27.99 -> 28.15 | Helped |
| replace single 8×8 drag embeddings with multi-scale embeddings (ablation) | PartRM · PartDrag-4D | PSNR: 26.87 -> 28.15 | Helped |
Surprising findings
- Using only the appearance-learning stage fails to move the drawer to the correct position, showing multi-view image supervision alone cannot learn part motion.
Limitations
- Struggles to generalize to articulated objects and drag patterns that deviate significantly from the training distribution.
- Relies on Zero123++ multi-view generation and drag propagation via Segment Anything, which can introduce errors.
- Drag magnitude estimation for propagated drags may be inaccurate.
Experimental setup
Datasets: PartDrag-4D PartNet-Mobility Objaverse-Animation-HQ
Benchmarks: PartDrag-4D Objaverse-Animation-HQ
Metrics: PSNR SSIM LPIPS inference time
Baselines: Puppet-Master DragAPart DiffEditor
Builds on: LGM Zero123++ Puppet-Master DragAPart Segment Anything
Closest papers by meaning
- Building Interactable Replicas of Complex Articulated Objects via Gaussian Splatting (ICLR 2025 · similarity 0.87)
- LongScape: Advancing Long-Horizon Embodied World Models with Context-Aware MoE (NeurIPS 2026 · similarity 0.87)
- PointZero: 3D Point Track Completion for Learning Transferable 3D Dynamics (NeurIPS 2026 · similarity 0.87)
- Learning Partonomic 3D Reconstruction from Image Collections (CVPR 2025 · similarity 0.87)
- Stable Part Diffusion 4D: Multi-View RGB and Kinematic Parts Video Generation (NeurIPS 2025 · similarity 0.87)
- DGS-LRM: Real-Time Deformable 3D Gaussian Reconstruction From Monocular Videos (NeurIPS 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.