Research map / 3D vision / Face and shape
Neural Implicit Representation for Building Digital Twins of Unknown Articulated Objects
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Builds digital twins of unknown articulated objects from two RGB-D scans at different joint states via per-object neural object fields and a two-stage segmentation/motion optimization with consistency, matching, and collision losses.
Problem
Reconstructing geometry, part segmentation, and joint articulation of unknown multi-part articulated objects from multi-view RGB-D observations at two articulation states, without category priors or training on articulated assets.
Approach
Decomposes the task into per-state neural SDF object reconstruction and then optimizes a point-level correspondence field derived from a part segmentation field and shared rigid part motions, supervised by SDF/color/occupancy consistency, LoFTR-based 2D matching, and a collision loss that penalizes multiple points mapping to one location.
Key results
- On the PARIS dataset, the method achieves average CD-w of 5.76 mm and Axis Ang Err of 10.11 degrees across simulation and real objects, versus 43.16 mm and 22.39 degrees for PARIS.
- On multi-part objects (Fridge-m, Storage-m), the method reduces CD-s and CD-m substantially relative to PARIS*-m, e.g., Fridge-m CD-w 0.89 vs 15.00.
- Ablations show removing the shared motion constraint raises Part Motion error from 0.09 to 0.28 and CD-m from 0.33 to 127.08 on multi-part objects.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use the proposed method instead of PARIS* | articulated-object reconstruction · PARIS synthetic All | Axis Ang Err (°): 11.14±16.1 -> 0.14±0.0 | Helped |
| Use the proposed method instead of PARIS* | articulated-object reconstruction · PARIS real All | CD-w (mm): 20.55±4.8 -> 5.76±0.0 | Helped |
| Add shared motion parameters across the two articulation models (ablation) | Ours · Multi-part objects | Axis Ang: 14.42±0.90 -> 0.26±0.04 | Helped |
| Add matching loss (ablation) | Ours · Multi-part objects | Part Motion: 8.06±0.06 -> 0.09±0.01 | Helped |
| Add RGB consistency loss (ablation) | Ours · Multi-part objects | Axis Ang: 0.43±0.11 -> 0.26±0.04 | Helped |
| Add occupancy consistency loss (ablation) | Ours · Multi-part objects | CD-m: 164.33±92.62 -> 0.33±0.00 | Helped |
| Add collision loss (ablation) | Ours · Multi-part objects | CD-m: 191.76±2.08 -> 0.33±0.00 | Helped |
| Add depth supervision to PARIS | PARIS · PARIS synthetic dataset | CD-w | Helped |
| Add depth supervision to PARIS | PARIS · PARIS synthetic dataset | articulation prediction | Hurt |
Surprising findings
- Adding depth supervision to PARIS (PARIS*) improves shape reconstruction but increases optimization failures and variance, worsening average articulation predictions.
Limitations
- Assumes camera poses are known and object base parts are aligned.
- Assumes the number of joints/parts is given.
- Fusing observations across states when occlusion patterns change is left open.
Experimental setup
Datasets: PARIS two-part object dataset PartNet-Mobility Shape2Motion
Benchmarks: PARIS two-part object dataset Synthetic Multi-Part Object Dataset
Metrics: Chamfer distance (CD-w, CD-s, CD-m) Axis Ang Err Axis Pos Err Part Motion Err
Baselines: Ditto PARIS PARIS* PARIS*-m CSG-reg 3Dseg-reg
Builds on: BundleSDF PARIS Ditto LoFTR Instant-NGP
Closest papers by meaning
- ArticulatedGS: Self-supervised Digital Twin Modeling of Articulated Objects using 3D Gaussian Splatting (CVPR 2025 · similarity 0.83)
- Learning Implicit Representation for Reconstructing Articulated Objects (ICLR 2024 · similarity 0.83)
- Articulate your NeRF: Unsupervised articulated object modeling via conditional view synthesis (NeurIPS 2024 · similarity 0.82)
- DIPO: Dual-State Images Controlled Articulated Object Generation Powered by Diverse Data (NeurIPS 2025 · similarity 0.82)
- Real2Code: Reconstruct Articulated Objects via Code Generation (ICLR 2025 · similarity 0.82)
- SINGAPO: Single Image Controlled Generation of Articulated Parts in Objects (ICLR 2025 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.