Research map / 3D vision / Gaussian splatting
Building Interactable Replicas of Complex Articulated Objects via Gaussian Splatting
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes ArtGS, which reconstructs interactable articulated objects from two-state multi-view RGB-D by optimizing canonical 3D Gaussians with coarse-to-fine initialization and a center-based skinning-style part assignment module.
Problem
Reconstructing part meshes and joint articulation parameters of multi-part articulated objects from RGB-D observations at two joint states, where existing methods fail to integrate information across states and scale poorly with part count.
Approach
Uses learnable canonical 3D Gaussians as a bridge between two object states, initialized by Hungarian matching of single-state Gaussians refined with motion analysis, and assigns Gaussians to parts via learnable centers with Mahalanobis distance and Gumbel-softmax, unsupervised, instead of relying on pretrained part priors.
Key results
- On PARIS, ArtGS achieves axis angle error of 0.02 on average vs 0.13 for DTA, and part motion error 0.02 vs 0.13.
- On ArtGS-Multi, ArtGS reduces axis angle error on Table-31249 to 0.04 from DTA's 20.62 while training in about 8 minutes vs 37 minutes for DTA.
- On PARIS, ArtGS trains in about 9 minutes per object versus about 29 minutes for DTA.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| canonical Gaussian coarse-to-fine initialization (vs random init) (ablation) | ArtGS · Ablation on Storage-47648 and Oven-101908 | Axis Ang and CD-m: Axis Ang 0.14 -> 24.15 without; CD-m 0.25 -> 1030.19 on Oven | Helped |
| center-based part assignment (vs MLP or slot-attention segmentation) (ablation) | ArtGS · Ablation on Storage-47648 and Oven-101908 | Axis Ang: Axis Ang 0.14 vs 21.84 (MLP) and 25.43 (SA) on Storage | Helped |
| motion-prior center initialization (cluster movable Gaussians only) vs clustering all Gaussians (ablation) | ArtGS · Ablation on Storage-47648 | CD-s: CD-s 0.67 full vs 258.23 without motion prior | Helped |
| joint-type prediction warm-up with type constraints (ablation) | ArtGS · Ablation on Storage-47648 and Oven-101908 | Axis Ang: Axis Ang 0.14 full vs 0.16 without (Storage); 0.04 vs 0.04 (Oven) | Helped |
| random part center initialization (vs motion-prior clustering) (ablation) | ArtGS · Ablation on Oven-101908 | CD-m: CD-m 0.25 full vs 8.86 random center init | Hurt |
| ArtGS vs DTA on PARIS | ArtGS · PARIS all objects | Axis Ang: 0.02 vs 0.13 | Helped |
| ArtGS vs DTA on PARIS | ArtGS · PARIS all objects | CD-w: 2.15 vs 1.79 (DTA) | Hurt |
| ArtGS vs DTA on PARIS | ArtGS · PARIS all objects | Part Motion: 1.94 vs 1.85 (DTA) | Hurt |
| ArtGS vs DTA on multi-part objects (DTA-Multi and ArtGS-Multi) | ArtGS · DTA-Multi and ArtGS-Multi | Axis Ang and training time: Axis Ang e.g. 0.02 vs 0.25 (Fridge-m); time 8 vs 32 min | Helped |
Surprising findings
- DTA performs well on whole-object and static-part Chamfer distance on simple objects but degrades sharply on low-visibility states, which ArtGS handles better.
- Even with ground-truth depth, TSDF-based mesh extraction does not beat the proposed pipeline on simple synthetic objects, so the CD-w gap there is limited by mesh extraction.
Limitations
- Limited to two object states, which may not capture complex multi-part articulation.
- Part assignment failures occur for parts with similar motions (e.g., parallel drawers), and initialization errors in part centers can cause mis-segmentation.
- Mesh quality is bounded by TSDF extraction from Gaussian-rendered depth.
Experimental setup
Datasets: PARIS DTA-Multi ArtGS-Multi PartNet-Mobility MultiScan
Benchmarks: PARIS DTA-Multi ArtGS-Multi
Metrics: Chamfer Distance (CD-w, CD-s, CD-m) Axis angle error Axis position error Part motion error training time PSNR SSIM LPIPS
Baselines: Ditto PARIS CSG-reg 3Dseg-reg DTA
Builds on: 3D Gaussian Splatting PARIS DTA SC-GS REACTO
Closest papers by meaning
- ArticulatedGS: Self-supervised Digital Twin Modeling of Articulated Objects using 3D Gaussian Splatting (CVPR 2025 · similarity 0.93)
- REArtGS: Reconstructing and Generating Articulated Objects via 3D Gaussian Splatting with Geometric and Motion Constraints (NeurIPS 2025 · similarity 0.90)
- GauHuman: Articulated Gaussian Splatting from Monocular Human Videos (CVPR 2024 · similarity 0.89)
- RigGS: Rigging of 3D Gaussians for Modeling Articulated Objects in Videos (CVPR 2025 · similarity 0.89)
- HUGS: Human Gaussian Splats (CVPR 2024 · similarity 0.88)
- Human Gaussian Splatting: Real-time Rendering of Animatable Avatars (CVPR 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.