Research map / 3D vision / Human pose estimation
3D-Aware Hypothesis & Verification for Generalizable Relative Object Pose Estimation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a hypothesis-and-verification framework that scores sampled relative rotations by applying them to learned 3D volumes lifted from two 2D feature maps, enabling single-reference generalizable relative object pose estimation.
Problem
Estimating relative object rotation between a single reference view and a query view of an unseen object under large pose variation, where dense reference views are unavailable.
Approach
Lift 2D feature maps of reference and query into 3D volumes via a transformer 3D reasoning module, sample rotation hypotheses, apply each as a 3D transform to the reference volume, and verify by feature similarity after projecting to orthogonal planes; the highest-scoring hypothesis is selected.
Key results
- On Objaverse (GROP), the method reaches 28.1 angular error and 58.4% Acc@15°, versus 33.5 and 42.9% for the next best (RelPose++).
- On LINEMOD (GROP), Acc@15° is 29.9% vs 15.8% for RelPose++; on CO3D, angular error 28.5 vs 38.5 for RelPose++.
- Replacing the 3D-aware verification with an energy-based model (RelPose*) drops LINEMOD Acc@15° from 29.9% to 7.9%.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use 3D-aware hypothesis-and-verification | relative object pose estimation · CO3D | Angular Error: 38.5 -> 28.5 | Helped |
| use 3D-aware hypothesis-and-verification | relative object pose estimation · CO3D | Acc @ 30° (%): 77.0 -> 83.5 | Helped |
| use 3D-aware hypothesis-and-verification | relative object pose estimation · CO3D | Acc @ 15° (%): 69.8 -> 71.0 | Helped |
| use 3D-aware hypothesis-and-verification | relative object pose estimation · Objaverse | Acc @ 15° (%): 42.9 -> 58.4 | Helped |
| use 3D-aware hypothesis-and-verification | relative object pose estimation · LINEMOD | Acc @ 15° (%): 15.8 -> 29.9 | Helped |
| add self-attention and cross-attention layers to the 3D reasoning blocks (ablation) | Ours · LINEMOD | Acc @ 15° (%): 28.2 -> 29.9 | Helped |
| add 3D masking (ablation) | Ours · LINEMOD | Acc @ 15° (%): 27.9 -> 29.9 | Helped |
| use 3D masking instead of 2D masking over RGB images (ablation) | Ours · LINEMOD | Acc @ 15° (%): 27.3 -> 29.9 | Helped |
| add the 2D feature aggregation module (ablation) | Ours · LINEMOD | Acc @ 15° (%): 26.4 -> 29.9 | Helped |
| replace 3D-aware verification with an energy-based model (ablation) | Ours with unchanged feature extraction backbone · LINEMOD | Acc @ 15° (%): 29.9 -> 7.9 | Hurt |
Surprising findings
- Feature-matching methods (SuperGlue, LoFTR, ZSP) degrade sharply as reference-query rotation grows, despite being strong on small variations.
Limitations
- Verification relies on local feature similarity and may be affected by occlusions.
- Evaluation assumes ground-truth bounding boxes for cropping, with robustness to box noise only tested on LINEMOD.
Experimental setup
Datasets: Objaverse LINEMOD CO3D
Benchmarks: CO3D GROP (Objaverse) GROP (LINEMOD)
Metrics: angular error Acc @ 30° Acc @ 15°
Baselines: SuperGlue LoFTR ZSP Regress RelPose RelPose++
Builds on: RelPose RelPose++ 6D continuous rotation representation infoNCE
Closest papers by meaning
- Structure-Aware Correspondence Learning for Relative Pose Estimation (CVPR 2025 · similarity 0.90)
- UNOPose: Unseen Object Pose Estimation with an Unposed RGB-D Reference Image (CVPR 2025 · similarity 0.90)
- NOPE: Novel Object Pose Estimation from a Single Image (CVPR 2024 · similarity 0.90)
- UA-Pose: Uncertainty-Aware 6D Object Pose Estimation and Online Object Completion with Partial References (CVPR 2025 · similarity 0.89)
- Sparse-view Pose Estimation and Reconstruction via Analysis by Generative Synthesis (NeurIPS 2024 · similarity 0.89)
- ImageNet3D: Towards General-Purpose Object-Level 3D Understanding (NeurIPS 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.