Research map / 3D vision / Human pose estimation
FAR: Flexible Accurate and Robust 6DoF Relative Camera Pose Estimation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes FAR, a Transformer that fuses learned end-to-end relative pose with a prior-guided RANSAC solver (weighted combination of solver and learned poses) for precise, robust 6DoF relative pose including translation scale.
Problem
Relative camera pose estimation between two wide-baseline images: correspondence+solver methods are precise but fragile under low overlap and cannot give translation scale, while direct regression is robust and scale-aware but less precise.
Approach
A Pose Transformer predicts a pose and per-component fusion weights (for rotation and translation) that linearly combine learned and solver poses; the learned pose is also used as a prior that re-weights RANSAC sampling and scoring, so the solver output is improved in data-scarce regimes.
Key results
- On Matterport3D, FAR reduces median translation error from 0.52 m (NOPE-SAC-Reg) to 0.25 m and mean rotation error from 9.66° (LoFTR) to 4.93°.
- On Map-free Relocalization, FAR (LoFTR) reaches 148 cm average median pose error and 18.1° rotation vs. 6D Reg's 168 cm and 22.5°.
- On StreetLearn large-overlap rotation, FAR achieves 1.81° mean error versus 2.43° for 8-Point ViT and 24.8° for LoFTR (median 1.81 vs 2.43 vs 24.8 from Table 4 values).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| combine learned pose estimates with solver output and prior-guided solving | LoFTR pose estimation · Matterport3D | mean rotation error (°): 9.66 -> 4.93 | Helped |
| combine learned pose estimates with solver output and prior-guided solving | NOPE-SAC-Reg pose estimation · Matterport3D | median translation error (m): 0.52 -> 0.25 | Helped |
| combine learned pose estimates with solver output and prior-guided solving | 8-Point ViT pose estimation · InteriorNet large overlap | mean rotation error (°): 2.90 -> 1.16 | Helped |
| combine learned pose estimates with solver output and prior-guided solving | 8-Point ViT pose estimation · StreetLearn small overlap | mean rotation error (°): 9.19 -> 7.89 | Helped |
| add LoFTR correspondences and FAR fusion | 6D Reg · Map-free Relocalization | average median translation error: 168cm -> 148cm | Helped |
| add LoFTR correspondences and FAR fusion | 6D Reg · Map-free Relocalization | average median rotation error: 22.5° -> 18.1° | Helped |
| fuse Transformer pose with solver output (ablation) | FAR · Matterport3D | mean rotation error (°): 9.94 -> 4.93 | Helped |
| add a second round with prior-guided solver output (ablation) | FAR · Matterport3D | mean rotation error (°): 5.08 -> 4.93 | Helped |
| fuse prior-guided solver output with Transformer output (ablation) | FAR · Matterport3D | mean rotation error (°): 5.35 -> 4.93 | Helped |
| use a correspondence-only Vanilla Transformer instead of dense 8-Point ViT features (ablation) | FAR · Matterport3D | median translation error (m): 0.25 -> 0.37 | Hurt |
| omit Transformer-predicted scaling of solver translation (ablation) | FAR · Matterport3D | median translation error (m): 0.25 -> 0.31 | Hurt |
| use separate weights for rotation and translation instead of one shared weight (ablation) | FAR · Matterport3D | mean translation error (m): 0.50 -> 0.49 | Helped |
Surprising findings
- Plain LoFTR correspondences generalize poorly to StreetLearn (median rotation 24.8° on large overlap), while regression methods remain usable.
Limitations
- Multiple components make the pipeline slower at inference (3.3 it/s on 10 1080Ti).
- Training on affluent Matterport3D homes may hurt performance on typical residences.
- Assumes known camera intrinsics.
Experimental setup
Datasets: Matterport3D InteriorNet StreetLearn Map-free Relocalization MegaDepth KITTI
Benchmarks: Matterport3D InteriorNet StreetLearn Map-free Relocalization
Metrics: median rotation error mean rotation error median translation error mean translation error percent within threshold AUC VCRE
Baselines: LoFTR SuperGlue 8-Point ViT NOPE-SAC-Reg PlaneFormers Sparse Planes Associative3D 6D Reg Cai et al. SIFT SuperPoint
Builds on: 8-Point ViT RANSAC LoFTR 6D rotation representation Arnold et al. Map-free relocalization regression
Closest papers by meaning
- From Correspondences to Pose: Non-minimal Certifiably Optimal Relative Pose without Disambiguation (CVPR 2024 · similarity 0.89)
- Relative Pose Estimation through Affine Corrections of Monocular Depth Priors (CVPR 2025 · similarity 0.88)
- Map-Relative Pose Regression for Visual Re-Localization (CVPR 2024 · similarity 0.87)
- 3D-Aware Hypothesis & Verification for Generalizable Relative Object Pose Estimation (ICLR 2024 · similarity 0.87)
- Co-op: Correspondence-based Novel Object Pose Estimation (CVPR 2025 · similarity 0.87)
- Pos3R: 6D Pose Estimation for Unseen Objects Made Easy (CVPR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.