Research map / 3D vision / Depth and stereo
Matching 2D Images in 3D: Metric Relative Pose from Metric Correspondences
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces MicKey, a keypoint matcher that predicts metric 3D keypoint coordinates in camera space and is trained end-to-end from relative pose supervision only, for metric relative pose estimation.
Problem
Estimating metric (scale-aware) relative camera pose between two images normally needs external depth estimators or depth measurements, since 2D-2D correspondences give pose only up to scale; strong depth supervision is also costly to obtain.
Approach
Regress 3D keypoint coordinates in camera space (2D offset plus depth on a patch grid) together with descriptors and confidences, and train the whole pipeline end-to-end with a differentiable RANSAC + Kabsch solver and a VCRE pose loss, so only image pairs with relative poses are needed (no depth, no overlap, no reconstructions).
Key results
- On the Map-free test set, MicKey (pose supervision only) reaches VCRE AUC 0.74 and precision 49.2%, the top VCRE results, versus RoMa at 0.67 / 45.6%.
- On ScanNet, MicKey reaches VCRE AUC 0.99 and precision 92.8%, comparable to fully supervised matchers such as LoFTR (0.99 / 91.3%).
- MicKey matching time is about 120 ms on Map-free, roughly 85% lower than RoMa (820 ms).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use MicKey instead of RoMa | Map-free metric relative pose estimation · Map-free test | AUC: 0.67 -> 0.74 | Helped |
| Use MicKey instead of RoMa | Map-free metric relative pose estimation · Map-free test | Precision (%): 45.6 -> 49.2 | Helped |
| Use MicKey instead of RoMa | ScanNet metric relative pose estimation · ScanNet test | AUC: 0.99 -> 0.99 | No clear effect |
| Use MicKey instead of RoMa | ScanNet metric relative pose estimation · ScanNet test | Precision (%): 94.4 -> 92.8 | Hurt |
| Add overlap supervision during training (ablation) | MicKey · Map-free test | AUC: 0.74 -> 0.75 | No clear effect |
| Add overlap supervision during training (ablation) | MicKey · Map-free test | Precision (%): 49.2 -> 49.2 | No clear effect |
| Add overlap supervision during training (ablation) | MicKey · ScanNet test | AUC: 0.99 -> 0.99 | No clear effect |
| Add overlap supervision during training (ablation) | MicKey · ScanNet test | Precision (%): 92.8 -> 93.7 | Helped |
| Replace DPT depth with MicKey depth (ablation) | SuperGlue · Map-free test | AUC: 0.60 -> 0.71 | Helped |
| Replace KBR depth with MicKey depth (ablation) | SuperGlue · Map-free test | AUC: 0.61 -> 0.71 | Helped |
| Replace DPT depth with MicKey depth (ablation) | LoFTR · Map-free test | AUC: 0.61 -> 0.71 | Helped |
| Replace KBR depth with MicKey depth (ablation) | LoFTR · Map-free test | AUC: 0.60 -> 0.71 | Helped |
Surprising findings
- MicKey trained with pose supervision only outperforms the overlap-plus-pose variant's precision slightly and performs comparably, suggesting overlap supervision adds little.
- MicKey attends to object shape in non-overlapping image pairs rather than low-level wall texture, a behavior not seen in other matchers.
Limitations
- For very fine accuracy thresholds, other methods give more accurate translation and rotation errors.
- DINOv2 features are limited in resolution, producing coarse depth and keypoint maps.
Experimental setup
Datasets: Map-free ScanNet
Benchmarks: Map-free Relocalisation benchmark ScanNet test split
Metrics: VCRE AUC VCRE precision at 90 px median translation error median rotation error matching time
Baselines: SIFT SiLK DISK DeDoDe SuperPoint-SuperGlue DISK-LightGlue LoFTR ASpanFormer RoMa RPR
Builds on: NG-DSAC DSAC SuperGlue DINOv2 Map-free relocalisation benchmark
Closest papers by meaning
- Structure-Aware Correspondence Learning for Relative Pose Estimation (CVPR 2025 · similarity 0.86)
- Relative Pose Estimation through Affine Corrections of Monocular Depth Priors (CVPR 2025 · similarity 0.86)
- From Correspondences to Pose: Non-minimal Certifiably Optimal Relative Pose without Disambiguation (CVPR 2024 · similarity 0.85)
- Practical Solutions to the Relative Pose of Three Calibrated Cameras (CVPR 2025 · similarity 0.85)
- UniDepth: Universal Monocular Metric Depth Estimation (CVPR 2024 · similarity 0.85)
- Rooms from Motion: Un-posed Indoor 3D Object Detection as Localization and Mapping (NeurIPS 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.