Research map / 3D vision / Human pose estimation
CatFormer: Category-Level 6D Object Pose Estimation with Transformer
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes CatFormer, a transformer-based RGB-D category-level 6D object pose estimator with coarse and fine point-cloud deformation modules, a graph module, and recurrent refinement, evaluated on CAMERA25, REAL275, and LINEMOD.
Problem
Category-level 6D object pose estimation suffers from intra-class shape variation and incomplete scene point clouds, so a fixed shape prior fits objects poorly and a single deformation is often insufficient.
Approach
Deform and complete the point cloud in feature space with transformer-based cross-attention modules (coarse on scene points, fine on the prior), add a transformer-style graph module after each deformation, and refine the prior point cloud recurrently over several iterations guided by fused RGB-point features.
Key results
- On REAL275, CatFormer reaches 47.7 on 5°2cm versus 46.5 for HS-Pose, and 83.1 IoU50 versus 82.1 for HS-Pose.
- On REAL275 at 10°2cm, CatFormer scores 69.0 versus HS-Pose's 68.6; on CAMERA25 the gains are only slight.
- On LINEMOD instance-level, CatFormer achieves 99.3 ADD(-S) at 8 FPS, versus 99.4 for PVN3D.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add CatFormer's coarse deformation module (ablation) | CatFormer · REAL275 | 5°2cm: 32.5 -> 47.7 | Helped |
| add CatFormer's fine deformation module (ablation) | CatFormer · REAL275 | 5°2cm: 38.9 -> 47.7 | Helped |
| add the graph module (ablation) | CatFormer · REAL275 | 5°2cm: 29.8 -> 47.7 | Helped |
| increase recurrent refinement from 1 to 5 iterations (ablation) | CatFormer · REAL275 | 5°2cm: 43.0 -> 47.7 | Helped |
| increase recurrent refinement from 3 to 5 iterations (ablation) | CatFormer · REAL275 | 5°2cm: 44.5 -> 47.7 | Helped |
| increase recurrent refinement from 4 to 5 iterations (ablation) | CatFormer · REAL275 | 5°2cm: 46.8 -> 47.7 | Helped |
| increase recurrent refinement from 5 to 6 iterations (ablation) | CatFormer · REAL275 | 5°2cm: 47.7 -> 46.2 | Hurt |
| increase recurrent refinement from 5 to 7 iterations (ablation) | CatFormer · REAL275 | 5°2cm: 47.7 -> 43.3 | Hurt |
| include the distribution loss (ablation) | CatFormer · REAL275 | 5°2cm | Helped |
| include the deformation loss (ablation) | CatFormer · REAL275 | 5°2cm | Helped |
| use the graph module with SPD | SPD · REAL275 | 5°2cm: 54.3 -> 61.4 | Helped |
| use the graph module with CR-Net | CR-Net · REAL275 | 5°2cm: 34.3 -> 32.9 | Hurt |
Surprising findings
- Point cloud completion gives little benefit on the synthetic CAMERA25 data, with gains concentrated on the real-world REAL275 dataset.
Limitations
- Gains over the previous SOTA on CAMERA25 are slight.
- Relies on Mask-RCNN for segmentation and RGB-D input, so it is not model-independent.
- Multi-iteration refinement degrades beyond five iterations.
Experimental setup
Datasets: CAMERA25 REAL275
Benchmarks: CAMERA25 REAL275 LINEMOD
Metrics: IoU25 IoU50 IoU75 5deg2cm 5deg5cm 10deg2cm 10deg5cm ADD(-S)
Baselines: NOCS SPD SGPA CR-Net FS-Net DualPoseNet SAR-Net GPV-Pose HS-Pose PVNet G2L-Net DenseFusion
Builds on: NOCS SPD SGPA PointNet++ PSPNet
Closest papers by meaning
- Rethinking Correspondence-based Category-Level Object Pose Estimation (CVPR 2025 · similarity 0.86)
- SecondPose: SE(3)-Consistent Dual-Stream Feature Fusion for Category-Level Pose Estimation (CVPR 2024 · similarity 0.86)
- CAP-Net: A Unified Network for 6D Pose and Size Estimation of Categorical Articulated Parts from a Single RGB-D Image (CVPR 2025 · similarity 0.86)
- GeoReF: Geometric Alignment Across Shape Variation for Category-level Object Pose Refinement (CVPR 2024 · similarity 0.86)
- KeyPose: Category-Level 6D Object Pose Estimation with Self-Adaptive Keypoints (AAAI 2025 · similarity 0.86)
- FoundationPose: Unified 6D Pose Estimation and Tracking of Novel Objects (CVPR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.