Research map / Point clouds / Registration and matching
AutoURDF: Unsupervised Robot Modeling from Point Cloud Frames Using Cluster Registration
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Builds URDF robot description files unsupervised from time-series point cloud frames by tracking 6-DoF point-cluster registration, then segmenting moving parts, inferring topology via MST, and estimating joints.
Problem
Robot description files (URDF) require substantial manual effort to create; existing robot self-modeling and articulated-object methods need sensorimotor data, ground-truth annotations, or cannot handle serially connected multi-DoF robot structures.
Approach
A lightweight PointNet-style cluster registration network (Step Model plus Anchor Model) tracks 6-DoF poses of K-means point clusters across frames; a motion correlation matrix segments them into links, an MST over cluster positions yields topology, and relative transforms give revolute joint parameters for direct URDF export.
Key results
- Mean CD 9.63 mm vs 16.45 mm for Reart across eight robots; mean TED 2.26 vs 5.68 (Reart) and 6.64 (MBS).
- Ablations: removing the position or orientation term raises mean TED to 5.92 and 7.25 respectively, versus 2.26 for the full method.
- Automatically predicts the correct DoF count for all eight robots, and runs registration in 50 s plus 12 s URDF construction on an NVIDIA 3090, versus 2,120 s for Reart.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace Reart with AutoURDF | robot registration · Validation set (8 robots) | CD: 16.45 ± 12.18 -> 9.63 ± 2.67 | Helped |
| replace Reart with AutoURDF | body topology inference · Validation set (8 robots) | TED: 5.68± 4.37 -> 2.26 ± 2.16 | Helped |
| add Anchor Model (ablation) | AutoURDF registration · Validation set (8 robots) | CD: 10.73 ± 2.89 -> 9.64 ± 2.69 | Helped |
| add Step Model (ablation) | AutoURDF registration · Validation set (8 robots) | CD: 9.68 ± 2.69 -> 9.64 ± 2.69 | Helped |
| add orientation distance term (ablation) | AutoURDF cluster correlation · Validation set (8 robots) | TED: 7.25 ± 6.14 -> 2.26 ± 2.16 | Helped |
| add position distance term (ablation) | AutoURDF cluster correlation · Validation set (8 robots) | TED: 5.92 ± 6.18 -> 2.26 ± 2.16 | Helped |
| use 5 input sequences instead of 1 | AutoURDF · Validation set (8 robots) | CDr: 22.13 ± 12.87 -> 18.42 ± 11.96 | Helped |
| use 5 input sequences instead of 1 | AutoURDF · Validation set (8 robots) | EJD: 5.24 ± 3.00 -> 3.46 ± 2.04 | Helped |
| use 5 input sequences instead of 1 | AutoURDF · Validation set (8 robots) | EJA: 6.50 ± 4.34 -> 3.13 ± 1.90 | Helped |
Surprising findings
- Using only the orientation term in the correlation matrix is misled by links with similar rotational motion, producing merged segments.
Limitations
- Collision-free motion only, so the URDFs lack dynamic properties such as mass and inertia.
- Struggles to segment complex structures like the OP3 humanoid with only 10 frames.
- Limited to revolute joints and tree-like kinematic structures.
Experimental setup
Datasets: simulated robot point cloud videos (8 robots) real-world WX200 scan point cloud video
Benchmarks: AutoURDF validation set (WX200, Franka Panda, UR5e, Bolt, Solo8, PhantomX, Allegro Hand, OP3)
Metrics: L1 Chamfer distance (CD) tree edit distance (TED) joint axis distance (EJD) joint angle error (EJA) repose Chamfer distance (CDr)
Baselines: Reart MultiBodySync NPP
Builds on: PointNet Chamfer distance marching cubes Silhouette score
Closest papers by meaning
- URDF-Anything: Constructing Articulated Objects with 3D Multimodal Language Model (NeurIPS 2025 · similarity 0.86)
- Point Cloud Matters: Rethinking the Impact of Different Observation Spaces on Robot Learning (NeurIPS 2024 · similarity 0.85)
- URDF-Anything+: End-to-End Generation for Simulation-Ready Articulated Assets (NeurIPS 2026 · similarity 0.85)
- PointMapPolicy: Structured Point Cloud Processing for Multi-Modal Imitation Learning (NeurIPS 2025 · similarity 0.84)
- Pre-training Auto-regressive Robotic Models with 4D Representations (ICML 2025 · similarity 0.83)
- PointZero: 3D Point Track Completion for Learning Transferable 3D Dynamics (NeurIPS 2026 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.