Research map / 3D vision / NeRF and radiance fields
Articulate your NeRF: Unsupervised articulated object modeling via conditional view synthesis
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes an unsupervised method that learns part segmentation and articulation of multi-part articulated objects from two multi-view observations by freezing a static NeRF and distilling part-aware rendering with a voxel-grid initialization and decoupled optimization.
Problem
Unsupervised estimation of part segmentation and articulation (axis and motion of rigid parts) of articulated objects from multi-view images of two different articulation states, without 3D shape, part labels, or articulation supervision.
Approach
Learn a single static NeRF from one articulation state, freeze it, and attach a segmentation head plus per-part rigid transforms applied via virtual rays to render the other articulation; a voxel-grid initialization from opacity differences and a decoupled alternating optimization (pose via 2D Chamfer distance, segmentation via photometric loss, voxel refinement) stabilize learning.
Key results
- On six PartNet-Mobility objects averaged over 5 runs, the method reaches mIoU of 0.94-0.99 on all objects except the stapler-free comparison, versus PARIS 0.16 on the stapler, and PSNR of 34.31 on stapler vs 24.36 for PARIS.
- Direction error e_d on the laptop drops to 0.33 degrees versus 0.68 for PARIS under the reproduced setting.
- With only 8 target views the method surpasses PARIS trained on 100 views in direction error and PSNR (34.00 vs 32.74).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| decoupled optimization plus voxel-grid initialization and iterative refinement (vs. joint optimization) (ablation) | articulation estimation on fridge · PartNet-Mobility fridge | PSNR: from 18.09 to 35.10 | Helped |
| add decoupled pose estimation (DP) without iterative refinement (ablation) | fridge articulation · PartNet-Mobility fridge | PSNR: 18.09 to 25.16 | Helped |
| add iterative refinement of voxel grid (IR) (ablation) | decoupled pose estimation on fridge · PartNet-Mobility fridge | e_d (deg): 2.38 to 0.54 | Helped |
| reduce number of target-observation views from 100 to 2 (ablation) | proposed method · fridge, varying target views | PSNR: 35.10 to 22.65 | Hurt |
| use 8 target views instead of 100 (ablation) | proposed method · fridge, varying target views | PSNR: 34.00 vs 35.10 (100 views); still above PARIS 100-view 32.74 | Mixed |
| replace PARIS joint optimization with the proposed static-NeRF + decoupled method | part segmentation and novel articulation rendering · PartNet-Mobility, 5 runs | mIoU and PSNR: e.g. stapler mIoU 0.16 to 0.98; PSNR 24.36 to 34.31; laptop PSNR slightly lower (30.31 vs 29.27) | Helped |
| replace PARIS with proposed method for pose estimation | revolute and prismatic joints · PartNet-Mobility, 5 runs | e_d direction error: lower on laptop, stapler, blade, fridge; higher on oven/ prismatic storage in reproduced comparison | Mixed |
Surprising findings
- The authors' reproduction of PARIS yields much worse results than the originally reported numbers, despite using the official code.
- PARIS fails to converge for the stapler and blade in all 5 runs in the reproduced setting.
Limitations
- Fails to segment very thin parts when rendering error is small (e.g., glasses temples, laptop screen).
- Pose estimation relies on geometry of moving parts and struggles with near-symmetric objects (e.g., folding chair flipped 180 degrees).
- Performance bounded by the pre-trained static NeRF and inherits implicit model limits such as transparency and noisy camera poses.
Experimental setup
Datasets: PartNet-Mobility
Benchmarks: PartNet-Mobility
Metrics: direction error e_d position error e_p geodesic/rotation error e_g translation error e_t PSNR mIoU
Baselines: PARIS
Builds on: NeRF PARIS proposal network (online distillation) Instant-NGP-style coarse-to-fine volume sampling
Closest papers by meaning
- SINGAPO: Single Image Controlled Generation of Articulated Parts in Objects (ICLR 2025 · similarity 0.88)
- Articulation in Prime: Primitive-Based Articulated Object Understanding from a Single Casual Video (NeurIPS 2026 · similarity 0.88)
- Learning Implicit Representation for Reconstructing Articulated Objects (ICLR 2024 · similarity 0.87)
- CAGE: Controllable Articulation GEneration (CVPR 2024 · similarity 0.87)
- Building Interactable Replicas of Complex Articulated Objects via Gaussian Splatting (ICLR 2025 · similarity 0.87)
- Real2Code: Reconstruct Articulated Objects via Code Generation (ICLR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.