Research map / 3D vision / Human pose estimation
Universal Features Guided Zero-Shot Category-Level Object Pose Estimation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a training-free zero-shot category-level 6-DoF pose estimator from RGB-D that combines iterative 2D universal-feature (DINOv2+SD) keypoint correspondences for coarse pose with 3D universal-feature (DGCNN)-guided pixel-level shape and pose refinement.
Problem
Category-level 6-DoF object pose estimation for unseen categories normally requires retraining or fine-tuning, and existing zero-shot approaches suffer from correspondence degradation under large pose differences and from shape differences within a category.
Approach
Coarse-to-fine pipeline: iteratively re-render the reference model under the updated pose to recompute 2D universal-feature correspondences (Umeyama+RANSAC), then refine pose and a deformable reference mesh by mask, Chamfer and a 3D universal-feature alignment loss, with no training on the target categories.
Key results
- On REAL275 unseen categories, the method reaches 30.61 at 5°2cm and 50.15 at 10°2cm, versus Zero-Pose's 21.36 and 43.84 at the same thresholds.
- On Wild6D, the method reaches 47.69 at 5°2cm and 59.46 at 10°2cm, versus Zero-Pose's 24.89 and 41.38.
- Runs at 3.83 s per frame, slower than Zero-Pose (0.97 s) and the supervised baselines (0.01 s).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| increase coarse-pose iterations from 1 to 2 (ablation) | Ours without pose refinement · REAL275 | 5°2cm and 5°5cm accuracy: 24.16/27.10 -> 28.56/30.94 | Helped |
| increase coarse-pose iterations from 1 to 2 (ablation) | Ours without pose refinement · Wild6D | 5°2cm and 5°5cm accuracy: 32.79/52.63 -> 42.71/57.58 | Helped |
| add pose refinement with shape deformation (ablation) | Ours, 2 iterations · REAL275 | 5°2cm and 5°5cm accuracy: 28.56/30.94 -> 30.35/33.13 | Helped |
| add pose refinement with shape deformation (ablation) | Ours, 2 iterations · Wild6D | 5°2cm and 5°5cm accuracy: 42.71/57.58 -> 47.48/60.37 | Helped |
| add universal alignment loss Lg (ablation) | Ours with pose refinement · REAL275 | 5°2cm and 5°5cm accuracy: 30.35/33.13 -> 30.61/33.23 | Helped |
| add universal alignment loss Lg (ablation) | Ours with pose refinement · Wild6D | 5°2cm and 5°5cm accuracy: 47.48/60.37 -> 47.69/60.58 | Helped |
| use DINOv2 + SD instead of DINOv1 features (ablation) | Coarse pose estimation without iteration or refinement · REAL275 | 5°2cm and 5°5cm accuracy: 21.34/23.03 -> 24.16/27.10 | Helped |
| use DINOv2 + SD instead of DINOv1 features (ablation) | Coarse pose estimation without iteration or refinement · Wild6D | 5°2cm and 5°5cm accuracy: 23.81/42.18 -> 32.79/52.63 | Helped |
| add SD to DINOv1 features (ablation) | Coarse pose estimation without iteration or refinement · REAL275 | 5°2cm accuracy: 21.34 -> 21.35 | No clear effect |
| add SD to DINOv1 features (ablation) | Coarse pose estimation without iteration or refinement · Wild6D | 5°2cm accuracy: 23.81 -> 23.68 | No clear effect |
| add all three 2D features instead of DINOv2 + SD (ablation) | Coarse pose estimation without iteration or refinement · REAL275 | 5°2cm and 5°5cm accuracy: 24.16/27.10 -> 23.36/25.02 | Hurt |
| add all three 2D features instead of DINOv2 + SD (ablation) | Coarse pose estimation without iteration or refinement · Wild6D | 5°2cm and 5°5cm accuracy: 32.79/52.63 -> 24.51/43.72 | Hurt |
Surprising findings
- Adding Stable Diffusion features to DINOv2 gave little gain on REAL275 alone, while the DINOv2+SD combination was chosen for its gains on Wild6D and its robustness to symmetric objects.
Limitations
- Computation time is a bottleneck (3.83 s per frame).
- Occlusion degrades target shape and pose accuracy.
- Relies on Mask R-CNN segmentation and retrieval of reference meshes from Objaverse, with some meshes needing watertight processing.
Experimental setup
Datasets: Objaverse REAL275 Wild6D CAMERA25 HouseCat6D
Benchmarks: REAL275 Wild6D HouseCat6D
Metrics: IOU_0.25 IOU_0.5 5°2cm 5°5cm 10°2cm 10°5cm inference speed
Baselines: DPDN VI-Net SPD Self-Pose Wild6D SSC-6D MegaPose Zero-Pose
Builds on: Zero-Pose DINOv2 Stable Diffusion features DGCNN Umeyama NOCS
Closest papers by meaning
- UNOPose: Unseen Object Pose Estimation with an Unposed RGB-D Reference Image (CVPR 2025 · similarity 0.89)
- SchemaPose: RGB-Based Category-Level Pose Estimation with Parametric Category Schema (NeurIPS 2026 · similarity 0.89)
- Unsupervised Learning of Category-Level 3D Pose from Object-Centric Videos (CVPR 2024 · similarity 0.88)
- SecondPose: SE(3)-Consistent Dual-Stream Feature Fusion for Category-Level Pose Estimation (CVPR 2024 · similarity 0.88)
- GIVEPose: Gradual Intra-class Variation Elimination for RGB-based Category-Level Object Pose Estimation (CVPR 2025 · similarity 0.87)
- NOPE: Novel Object Pose Estimation from a Single Image (CVPR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.