Research map / 3D vision / 3D generation
Category-Agnostic Neural Object Rigging
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Learns a category-agnostic neural rigging representation by encoding deforming 3D shapes into sparse feature-embedded anisotropic blobs plus instance features, enabling pose editing by dragging blobs.
Problem
Automatically discovering low-dimensional rigging (pose) representations for deformable 4D objects across arbitrary categories without category-specific expert knowledge, correspondences, or priors.
Approach
Encode a point cloud into a sparse set of learnable anisotropic blobs (pose params from one frame, identity params from another during training) with a shared codebook cross-attention, voxelize blobs into an augmented feature volume, and decode occupancy with a transformer for re-posing.
Key results
- Achieves IoU 0.937 on DeformingThings4D vs. 0.875 for the best baseline (NeuralDeformationGraph).
- Achieves IoU 0.960 on FaMoS and 0.860 on Fish, outperforming all baselines by a large margin on nearly all datasets.
- Removing identity conditioning drops DeformingThings4D IoU from 0.937 to 0.853.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use CANOR instead of NeuralDeformationGraph | DeformingThings4D re-posing · DeformingThings4D | IoU: 0.875 -> 0.937 | Helped |
| Use CANOR instead of KeypointDeformer | FaMoS re-posing · FaMoS | IoU: 0.923 -> 0.960 | Helped |
| Use CANOR instead of SkeRig | Fish re-posing · Fish | IoU: 0.782 -> 0.860 | Helped |
| Use CANOR instead of NeuralDeformationGraph | Refrigerator re-posing · Refrigerator | IoU: 0.869 -> 0.903 | Helped |
| Use CANOR instead of NeuralDeformationGraph | Eyeglasses re-posing · Eyeglasses | IoU: 0.791 -> 0.770 | Hurt |
| Add identity conditioning (ablation) | CANOR · DeformingThings4D | IoU: 0.853 -> 0.937 | Helped |
| Use anisotropic blobs instead of isotropic blobs (ablation) | CANOR · DeformingThings4D | IoU: 0.934 -> 0.937 | Helped |
| Increase blob count from 8 to 24 (ablation) | CANOR · DeformingThings4D | IoU: 0.845 -> 0.937 | Helped |
| Increase blob count from 16 to 24 (ablation) | CANOR · DeformingThings4D | IoU: 0.927 -> 0.937 | Helped |
Surprising findings
- Isotropic blobs (w/o anisotropic blobs) reach IoU 0.934, close to the full model's 0.937.
Limitations
- Number of blobs must be upper-bounded per category (8 to 24 chosen manually).
- Baselines like SkeRig require dense correspondences which are estimated via NDP, and some baselines need hours of per-sequence optimization.
- Training requires paired frames of the same identity under different poses.
Experimental setup
Datasets: DeformingThings4D FaMoS Fish (Sketchfab) Shape2Motion (Refrigerator, Eyeglasses) clay-monster (custom iPhone scans)
Benchmarks: DeformingThings4D FaMoS Fish Shape2Motion Refrigerator Shape2Motion Eyeglasses
Metrics: IoU Chamfer distance L1 Chamfer distance L2
Baselines: KeypointDeformer NeuralDeformationGraph SkeRig
Builds on: Point Transformer Neural Deformation Graphs BlobGAN Shape auto-encoding transformer (3DShape2VecSet-style)
Closest papers by meaning
- Autonomous Manipulation Learning for Similar Deformable Objects via Only One Demonstration (CVPR 2023 · similarity 0.87)
- NeuROK: Generative 4D Neural Object Kinematics (CVPR 2026 · similarity 0.87)
- DNF: Unconditional 4D Generation with Dictionary-based Neural Fields (CVPR 2025 · similarity 0.85)
- Neural Pose Representation Learning for Generating and Transferring Non-Rigid Object Poses (NeurIPS 2024 · similarity 0.85)
- Dynamic Neural Surfaces for Elastic 4D Shape Representation and Analysis (CVPR 2025 · similarity 0.84)
- Implicit Modeling of Non-rigid Objects with Cross-Category Signals (AAAI 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.