Research map / 3D vision / Human pose estimation
HouseCat6D - A Large-Scale Multi-Modal Category Level 6D Object Perception Dataset with Household Objects in Realistic Scenarios
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces HouseCat6D, a large-scale multi-modal (RGBD+polarimetric) category-level 6D object pose and grasp dataset of household objects, with tracker-based high-accuracy annotations, and benchmarks NOCS, FS-Net, GPV-Pose, VI-Net and KGN on it.
Problem
Category-level 6D object pose estimation lacks datasets that combine large scale, high-accuracy annotation, realistic scenes, pose variety, and challenging photometric materials.
Approach
Acquire multi-modal RGBD+polarimetric sequences with a handheld rig tracked by an external infrared tracker (no checkerboards), with improved hand-eye calibration, timestamp sync, and bundle-adjustment refinement, yielding mm-level pose labels and dense viewpoint coverage plus parallel-jaw grasp labels.
Key results
- Annotation quality RMSE of 1.35 mm to 1.73 mm, versus 0.80 mm for PhoCaL and 3.4 mm for StereOBJ-1M
- VI-Net achieves the best category-level baseline result with 85.2 3D25 and 62.4 3D50 mean IoU on the HouseCat6D test split
- Grasp coverage rate for KGN improves with keypoint refinement, e.g. Glass from 16.3% to 40.8%
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use VI-Net instead of NOCS | HouseCat6D test pose estimation · HouseCat6D test | 3D IoU (25% / 50%): 50.0 / 21.2 -> 80.7 / 56.4 | Helped |
| use ground-truth NOCS maps with sensor depth instead of predicted NOCS maps (ablation) | NOCS · HouseCat6D test | 3D IoU (25% / 50%): 50.0 / 21.2 -> 96.7 / 93.6 | Helped |
| train on a reduced set matching PhoCaL-like pose coverage (ablation) | VI-Net · HouseCat6D test | 3D IoU (25% / 50%): 80.7 / 56.4 -> 68.4 / 31.3 | Hurt |
| add keypoint refinement (ablation) | KGN · HouseCat6D test grasping (Bottle, Can, Cup, Glass, Remote, Tube) | GCR (%): 17.4 / 35.4 / 32.6 / 16.3 / 64.3 / 47.3 -> 24.1 / 58.3 / 34.1 / 40.8 / 64.5 / 61.1 | Helped |
| add keypoint refinement (ablation) | KGN · HouseCat6D test grasping (Bottle, Can, Cup, Glass, Remote, Tube) | OSR (%): 97.8 / 75.2 / 100 / 100 / 92.1 / 100 -> 99.8 / 87.6 / 100 / 100 / 94.8 / 100 | Helped |
| add keypoint refinement (ablation) | KGN · Real-world grasp test (Box, Cup, Glass, Remote, Unknown) | GSR (%): 80.0 / 66.7 / 66.7 / 53.3 / 53.3 -> 80.0 / 66.7 / 73.3 / 33.3 / 60.0 | Mixed |
Surprising findings
- Using ground-truth NOCS maps with sensor depth lifting drops performance sharply (mean 3D50 falls to 22.6%), showing sensor depth noise dominates 3D lifting errors
- KGN trained on HouseCat6D grasps transparent glass objects well, unlike its original synthetic-trained performance
Limitations
- Annotation accuracy is lower than robotic-arm-based PhoCaL annotations
- Current category-level methods handle occlusions poorly, leaving substantial room for improvement
- Glass and tube suffer notably from noisy active stereo depth
Experimental setup
Datasets: HouseCat6D PhoCaL NOCS StereOBJ-1M Wild6D TOD
Benchmarks: HouseCat6D
Metrics: 3D IoU at 25% and 50% point-wise RMSE grasp coverage rate object success rate real-world grasp success rate
Baselines: NOCS FS-Net GPV-Pose VI-Net KGN
Builds on: NOCS PhoCaL GraspNet-1Billion KGN
Closest papers by meaning
- CAP-Net: A Unified Network for 6D Pose and Size Estimation of Categorical Articulated Parts from a Single RGB-D Image (CVPR 2025 · similarity 0.87)
- SchemaPose: RGB-Based Category-Level Pose Estimation with Parametric Category Schema (NeurIPS 2026 · similarity 0.86)
- Vision Foundation Model Enables Generalizable Object Pose Estimation (NeurIPS 2024 · similarity 0.86)
- SingRef6D: Monocular Novel Object Pose Estimation with a Single RGB Reference (NeurIPS 2025 · similarity 0.85)
- Any6D: Model-free 6D Pose Estimation of Novel Objects (CVPR 2025 · similarity 0.85)
- Open-Vocabulary Object 6D Pose Estimation (CVPR 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.