Research map / 3D vision / Human pose estimation
Structure-Aware Correspondence Learning for Relative Pose Estimation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a structure-aware correspondence learning method that regresses 3D-3D keypoint correspondences for relative rotation estimation without explicit feature matching, trained with a keypoint reconstruction loss.
Problem
Relative pose estimation for unseen objects from a single reference image suffers from small visible overlaps and unreliable 3D feature estimation in invisible regions for existing correspondence-based methods.
Approach
Learnable query-based keypoint extraction supervised by a keypoint-based image reconstruction loss, followed by self- and cross-attention over keypoints with rotary positional encoding to directly regress 3D correspondences across query and reference coordinate frames, solved by weighted SVD.
Key results
- On CO3D, mAE 14.2 vs 19.9 for DVMNet (5.7 degree reduction), Acc@15 80.2%.
- On Objaverse, mAE 15.3 and Acc@15 74.0, outperforming DVMNet (20.2, 57.2).
- On LineMOD, mAE 27.2 and Acc@15 41.8, outperforming DVMNet (36.8, 23.8).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use structure-aware keypoint extraction instead of dense pixel features (ablation) | Structure-Aware Correspondence Learning · CO3D | mAE: 15.52 -> 14.2 | Helped |
| use structure-aware keypoint extraction instead of randomly sampled points (ablation) | Structure-Aware Correspondence Learning · CO3D | mAE: 20.15 -> 14.2 | Helped |
| add self-attention (ablation) | Structure-Aware Correspondence Estimation · CO3D | mAE: 17.79 -> 14.2 | Helped |
| add cross-attention (ablation) | Structure-Aware Correspondence Estimation · CO3D | mAE: 18.53 -> 14.2 | Helped |
| regress 3D keypoint coordinates instead of using dense regression (ablation) | Structure-Aware Correspondence Learning · CO3D | mAE: 19.37 -> 14.2 | Helped |
| regress 3D keypoint coordinates instead of regressing relative pose from global features (ablation) | Structure-Aware Correspondence Learning · CO3D | mAE: 21.97 -> 14.2 | Helped |
| use mask loss (ablation) | Structure-Aware Correspondence Learning · CO3D | mAE: 16.37 -> 14.2 | Helped |
| use confidence score estimation (ablation) | Structure-Aware Correspondence Learning · CO3D | mAE: 15.58 -> 14.2 | Helped |
| vary the keypoint count from 16 to 32 to 48 to 64 (ablation) | Structure-Aware Correspondence Learning · CO3D | mAE: 18.03, 15.95, 14.2, 14.35 | Mixed |
Surprising findings
- Regressing 3D keypoint coordinates is more effective than directly regressing the global relative rotation, per the ablation.
Limitations
- Relies on ground-truth bounding box cropping of objects.
- Only predicts rotation; translation assumed handled by existing 2D detection.
Experimental setup
Datasets: CO3D Objaverse LineMOD
Benchmarks: CO3D Objaverse LineMOD
Metrics: mean angular error Acc@30 Acc@15
Baselines: SuperGlue LoFTR ZSP RelPose RelPose++ 3DAHV DVMNet
Builds on: DVMNet CroCo RoPE weighted SVD
Closest papers by meaning
- 3D-Aware Hypothesis & Verification for Generalizable Relative Object Pose Estimation (ICLR 2024 · similarity 0.90)
- From Correspondences to Pose: Non-minimal Certifiably Optimal Relative Pose without Disambiguation (CVPR 2024 · similarity 0.89)
- Rethinking Correspondence-based Category-Level Object Pose Estimation (CVPR 2025 · similarity 0.89)
- UNOPose: Unseen Object Pose Estimation with an Unposed RGB-D Reference Image (CVPR 2025 · similarity 0.88)
- Instance-Adaptive and Geometric-Aware Keypoint Learning for Category-Level 6D Object Pose Estimation (CVPR 2024 · similarity 0.87)
- Co-op: Correspondence-based Novel Object Pose Estimation (CVPR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.