Research map / 3D vision / Human pose estimation

SecondPose: SE(3)-Consistent Dual-Stream Feature Fusion for Category-Level Pose Estimation

CVPR 2024 · 3D vision › Human pose estimation · approach: Category-level object pose estimation · 37 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes SecondPose, which fuses hierarchical SE(3)-invariant point-pair geometric features with DINOv2 semantic features into an SE(3)-consistent representation for category-level 6D pose and size estimation from RGB-D.

Problem

Category-level 9-DoF pose estimation struggles with large intra-class shape variation, and mean-shape priors fail to capture it; no prior work directly fuses DINOv2 semantic features with object-specific geometry for pose.

Approach

Hierarchical-panel point pair features (HP-PPF) computed over multi-scale neighborhoods are SE(3)-invariant geometric descriptors; these are point-aligned with DINOv2 patch features and RGB in a spherical feature map, fused by per-stream ResNets, and fed to VI-Net's rotation regressor, making the pose-estimation mapping approximately rotation-invariant before canonical-space regression.

Key results

• On NOCS-REAL275, SecondPose reaches 56.2% at 5°2cm and 86.0% at 10°5cm, versus VI-Net's 50.0% and 82.1%.
• On HouseCat6D, SecondPose averages about 9.7% higher IoU50 than the second-best method VI-Net.
• Ablation: removing semantic features drops 5°2cm accuracy by 5.1 points; removing geometric features drops it by 1.1 points; removing both drops it by 6.3 points.

Measured effects reported in the paper

Surprising findings

• Removing geometric features alone causes only a small drop (1.1 points at 5°2cm), while removing semantic features causes a much larger drop.
• The method's efficacy is limited when DINOv2 fails to provide meaningful semantic information for specific images.

Limitations

• Performance is bounded by DINOv2 features; when DINOv2 gives no meaningful semantics for an image, the method cannot overcome this.
• Relies on Mask R-CNN segmentation and a separate translation-size network.
• Rotation is only approximately SE(3)-invariant (approximate invariance shown empirically in Fig. 4).

Experimental setup

Datasets: NOCS-REAL275 NOCS-CAMERA25

Benchmarks: NOCS-REAL275 HouseCat6D

Metrics: mAP 5°2cm mAP 5°5cm mAP 10°2cm mAP 10°5cm 3D IoU mAP at 75% 3D IoU mAP at 25% 3D IoU mAP at 50%

Baselines: VI-Net DPDN SPD + CATRE HS-Pose IST-Net GPV-Pose FS-Net DualPoseNet

Builds on: VI-Net Point Pair Features DINOv2 NOCS

Closest papers by meaning

• Rethinking Correspondence-based Category-Level Object Pose Estimation (CVPR 2025 · similarity 0.88)
• SchemaPose: RGB-Based Category-Level Pose Estimation with Parametric Category Schema (NeurIPS 2026 · similarity 0.88)
• Learning Shape-Independent Transformation via Spherical Representations for Category-Level Object Pose Estimation (ICLR 2025 · similarity 0.88)
• Dual Pose-invariant Embeddings: Learning Category and Object-specific Discriminative Representations for Recognition and Retrieval (CVPR 2024 · similarity 0.88)
• Universal Features Guided Zero-Shot Category-Level Object Pose Estimation (AAAI 2025 · similarity 0.88)
• GCE-Pose: Global Context Enhancement for Category-level Object Pose Estimation (CVPR 2025 · similarity 0.87)