Research map / 3D vision / Human pose estimation
Reloc3r: Large-Scale Training of Relative Camera Pose Regression for Generalizable, Fast, and Accurate Visual Localization
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Reloc3r trains a fully symmetric relative camera pose regression network (DUSt3R-initialized ViT) on ~8M image pairs and combines it with a training-free motion averaging module for fast, generalizable visual localization.
Problem
Visual localization that is simultaneously generalizable to novel scenes, fast at test time, and accurate; existing absolute pose regression is scene-specific and relative pose regression generalizes poorly and lags in accuracy.
Approach
A fully symmetric ViT encoder-decoder with shared weights and a pose head that regresses only relative rotation and translation direction (angles, scale-free), trained at large scale on diverse datasets, followed by a parameter-free motion averaging step (median/quaternion rotation averaging and SVD-based camera center triangulation) over retrieved database pairs.
Key results
- Reloc3r-512 reaches 34.79 AUC@5 on ScanNet1500 versus 28.90 for ROMA and 23.81 for DUSt3R, running at about 25 ms per pair.
- On 7 Scenes, Reloc3r-512 averages 0.04 m / 1.02 degrees median error, outperforming all relative pose regression methods including those trained on the same dataset, without per-scene training.
- On CO3Dv2 multi-view relative pose, Reloc3r-512 achieves RRA@15 95.8, RTA@15 93.7, and mAA@30 82.9, the best among compared methods.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add the fully symmetric shared-weight architecture instead of asymmetric branches (ablation) | Reloc3r-512 · ScanNet1500 | AUC@5: 32.71 -> 34.79 | Helped |
| add the fully symmetric shared-weight architecture instead of asymmetric branches (ablation) | Reloc3r-512 · ScanNet1500 | AUC@10: 56.84 -> 58.37 | Helped |
| add the fully symmetric shared-weight architecture instead of asymmetric branches (ablation) | Reloc3r-512 · ScanNet1500 | AUC@20: 74.63 -> 75.56 | Helped |
| use direction-only translation supervision rather than metric-scale pose prediction (ablation) | Reloc3r-512 · ScanNet1500 | AUC@5: 25.70 -> 34.79 | Helped |
| use direction-only translation supervision rather than metric-scale pose prediction (ablation) | Reloc3r-512 · ScanNet1500 | AUC@10: 50.20 -> 58.37 | Helped |
| use direction-only translation supervision rather than metric-scale pose prediction (ablation) | Reloc3r-512 · ScanNet1500 | AUC@20: 70.07 -> 75.56 | Helped |
| increase input width from 224 to 512 pixels | Reloc3r · ScanNet1500 | AUC@5: 28.34 -> 34.79 | Helped |
Surprising findings
- Reloc3r's cross-attention layers learn patch-level correspondences even though training uses only pose supervision.
- An asymmetric variant of the network performs worse than the fully symmetric default while needing more compute.
Limitations
- Degeneracy when the query and all retrieved database images are perfectly collinear, making the metric scale unsolvable by motion averaging.
- Relies on off-the-shelf NetVLAD retrieval and uses top-10 pairs without spatial filtering.
Experimental setup
Datasets: CO3Dv2 ScanNet++ ARKitScenes BlendedMVS MegaDepth DL3DV RealEstate10K ScanNet1500 ACID 7 Scenes Cambridge Landmarks
Benchmarks: ScanNet1500 RealEstate10K ACID CO3Dv2 7 Scenes Cambridge Landmarks
Metrics: AUC@5/10/20 RRA@15 RTA@15 mAA@30 median translation/rotation error inference time
Baselines: DUSt3R MASt3R NoPoSplat ROMA Efficient LoFTR Map-free ExReNet Relative PN ImageNet+NCM PoseDiffusion VGGSfM RayDiffusion
Builds on: DUSt3R NetVLAD Map-free relocalization ExReNet RoPE
Closest papers by meaning
- Map-Relative Pose Regression for Visual Re-Localization (CVPR 2024 · similarity 0.90)
- HyperPose: Hypernetwork-Infused Camera Pose Localization and an Extended Cambridge Landmarks Dataset (CVPR 2025 · similarity 0.87)
- FAR: Flexible Accurate and Robust 6DoF Relative Camera Pose Estimation (CVPR 2024 · similarity 0.87)
- Pow3R: Empowering Unconstrained 3D Reconstruction with Camera and Scene Priors (CVPR 2025 · similarity 0.87)
- 3D-Aware Hypothesis & Verification for Generalizable Relative Object Pose Estimation (ICLR 2024 · similarity 0.86)
- ConDo: Continual Domain Expansion for Absolute Pose Regression (AAAI 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.