Research map / 3D vision / Face and shape
Learning Implicit Representation for Reconstructing Articulated Objects
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Learns an implicit skeleton (bones, skinning weights, rigidity coefficients, transformations) alongside explicit mesh/camera representations from monocular videos without 3D supervision, via an EM-like SIOS^2 algorithm.
Problem
Reconstructing 3D shape and motion of articulated objects from monocular or multi-view videos without category-specific skeletal templates or 3D supervision.
Approach
Treats an articulated object as an unknown semi-rigid skeleton covered by nonrigid skin; learns skeleton via mesh contraction initialization and iterative refinement guided by optical-flow-based bone motion and bone-length constancy, plus a Dynamic Rigidity loss that replaces ARAP with entropy-based per-edge rigidity coefficients.
Key results
- On DAVIS keypoint transfer, LIMR averages 80.2 PCK-T vs 71.9 for LASR and 74.1 for ViSER.
- On PlanetZoo, LIMR averages 69.9 PCK-T vs 66.9 for LASR.
- On AMA swing and samba, LIMR reduces average Chamfer distance to 12.9 cm from 14.6 cm for BANMo while using 19 bones instead of 25.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add the learned skeleton and implicit representation | LIMR compared with LASR · DAVIS | 2D keypoint transfer accuracy: 71.9 -> 80.2 | Helped |
| Add the learned skeleton and implicit representation | LIMR compared with LASR · PlanetZoo | 2D keypoint transfer accuracy: 66.9 -> 69.9 | Helped |
| Add Dynamic Rigidity instead of ARAP (ablation) | LIMR · DAVIS | 2D keypoint transfer accuracy: 78.5 -> 80.2 | Helped |
| Add Dynamic Rigidity instead of ARAP (ablation) | LIMR · PlanetZoo | 2D keypoint transfer accuracy: 68.2 -> 69.9 | Helped |
| Use the learned skeleton and implicit representation | LIMR compared with BANMo · AMA-samba | Chamfer Distance (cm): 15.3 -> 13.1 | Helped |
| Use the learned skeleton and implicit representation | LIMR compared with BANMo · AMA average | Chamfer Distance (cm): 14.6 -> 12.9 | Helped |
| Use the learned skeleton and implicit representation | LIMR compared with BANMo · AMA average | F@2%: 53.9 -> 55.8 | Helped |
| Use the learned skeleton and implicit representation | LIMR compared with BANMo · AMA average | mIoU: 61.8 -> 62.5 | Helped |
| Vary the skeleton-merging threshold t_o between 0.99 and 0.85 (ablation) | LIMR · reconstruction results | reconstruction performance | No clear effect |
Surprising findings
- RigNet, despite 3D supervision, produces an excessive number of unnecessary bones on low-motion regions such as the camel torso.
- ViSER performs poorly on PlanetZoo due to large camera motion, and BANMo fails to produce satisfactory results from single short videos.
Limitations
- Starts from a sphere rather than a template, so it needs multiple diverse views for best results.
- Continuously moving cameras cause camera-parameter errors that propagate into shape estimates.
- Requires long training (10-20 hours per run on one A100).
Experimental setup
Datasets: DAVIS BADJA PlanetZoo AMA BANMo casual videos
Benchmarks: BADJA PlanetZoo AMA swing and samba
Metrics: PCK-T (2D keypoint transfer accuracy) 3D Chamfer distance F-score mIoU
Baselines: LASR ViSER BANMo RigNet
Builds on: LASR BANMo ViSER As-Rigid-As-Possible loss
Closest papers by meaning
- REACTO: Reconstructing Articulated Objects from a Single Video (CVPR 2024 · similarity 0.91)
- Learning Monocular 3D Reconstruction of Articulated Categories From Motion (CVPR 2021 · similarity 0.91)
- SAMoR: Motion Modelling for Articulated Objects of Any Skeleton and Topology (NeurIPS 2026 · similarity 0.90)
- Articulation in Prime: Primitive-Based Articulated Object Understanding from a Single Casual Video (NeurIPS 2026 · similarity 0.90)
- RigGS: Rigging of 3D Gaussians for Modeling Articulated Objects in Videos (CVPR 2025 · similarity 0.89)
- S3O: A Dual-Phase Approach for Reconstructing Dynamic Shape and Skeleton of Articulated Objects from Single Monocular Video (ICML 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.