Research map / 3D vision / Depth and stereo
Map-Relative Pose Regression for Visual Re-Localization
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes marepo, a transformer-based absolute pose regressor that takes scene-coordinate maps from a fast per-scene ACE regressor plus camera intrinsics and is trained once across 450 Map-Free scenes, with optional minutes-long per-scene fine-tuning.
Problem
Absolute pose regression is accurate only with vast per-scene training data that is costly to create via novel view synthesis over days, and it generalizes poorly to unseen views.
Approach
Condition a scene-agnostic transformer pose regressor on predicted scene coordinate maps (from a fast per-scene SCR network) and camera-intrinsics-aware dynamic positional encoding, so the regressor learns a generic map-to-pose relation and outputs metric poses directly without RANSAC.
Key results
- On 7-Scenes, marepo averages 3.9cm / 1.68° median error, and marepoS (fine-tuned) reaches 3.2cm / 1.54°, versus 2.8cm / 0.93° for ACE and roughly 8-44cm for prior APR methods.
- On Wayspots, marepo achieves 47.2% of frames within 10cm/5° versus 5.3% for PoseNet, and 47.9% for marepoS, close to ACE's 52.2%.
- marepo runs at about 55.6 fps, faster than DSAC* and ACE (17.9 fps) on Wayspots.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Fine-tune M for each scene | marepo · 7-Scenes average | median translation error (cm): 3.9 -> 3.2 | Helped |
| Fine-tune M for each scene | marepo · Wayspots average | frames below 10cm/5°: 47.2% -> 47.9% | Mixed |
| Use marepo instead of ACE | Wayspots · Wayspots average | frames below 0.5m/5°: 62.6% -> 63.4% | Helped |
| Add Re-Attention (ablation) | marepo · Wayspots | frames below 5cm/5°: 10.9% -> 16.6% | Helped |
| Add Re-Attention (ablation) | marepo · Wayspots | frames below 10cm/5°: 28.3% -> 39.6% | Helped |
| Add Dynamic P.E. (ablation) | marepo · Wayspots | frames below 5cm/5°: 3.9% -> 16.6% | Helped |
| Add Dynamic P.E. (ablation) | marepo · Wayspots | frames below 10cm/5°: 18.6% -> 39.6% | Helped |
| Increase transformer blocks from 4 to 8 (dmodel 128) (ablation) | marepo · Wayspots | frames below 10cm/5°: 5.8% -> 14.7% | Helped |
| Increase transformer blocks from 8 to 12 (dmodel 128) (ablation) | marepo · Wayspots | frames below 10cm/5°: 14.7% -> 16.6% | Helped |
| Increase dmodel from 128 to 256 with 12 transformer blocks (ablation) | marepo · Wayspots | frames below 10cm/5°: 16.6% -> 19.0% | Helped |
| Train the full network from scratch per scene instead of training M across scenes (ablation) | marepo · Wayspots | frames below 50cm/5°: 63.4% -> 6.0% | Hurt |
| Train M from scratch per scene on pretrained GS instead of using the scene-agnostic pretrained M (ablation) | marepo · Wayspots | frames below 50cm/5°: 63.4% -> 18.7% | Hurt |
Surprising findings
- Training the full network end-to-end from scratch per scene performs extremely poorly (0.7% within 10cm/5° on Wayspots), showing the scene-agnostic training across hundreds of scenes is essential.
Limitations
- Relies on an external scene-specific SCR network (ACE) for scene coordinates, so the pipeline is not fully scene-agnostic end-to-end.
- Larger model variants (dmodel=512, 16/20 blocks) were not explored due to GPU and time cost.
Experimental setup
Datasets: Map-Free Dataset 7-Scenes Wayspots
Benchmarks: 7-Scenes Wayspots
Metrics: median translation error (cm) median rotation error (°) percentage of frames within 10cm/5° percentage of frames within 0.5m/5° throughput (fps)
Baselines: DSAC* ACE PoseNet PN Learn σ2 geo. PN LSTM PN Hourglass PN BranchNet MapNet Direct-PN MS-Transformer DFNet
Builds on: ACE LoFTR PoseNet Linear Transformer
Closest papers by meaning
- Scene-agnostic Pose Regression for Visual Localization (CVPR 2025 · similarity 0.90)
- Reloc3r: Large-Scale Training of Relative Camera Pose Regression for Generalizable, Fast, and Accurate Visual Localization (CVPR 2025 · similarity 0.90)
- ConDo: Continual Domain Expansion for Absolute Pose Regression (AAAI 2025 · similarity 0.90)
- FAR: Flexible Accurate and Robust 6DoF Relative Camera Pose Estimation (CVPR 2024 · similarity 0.87)
- HyperPose: Hypernetwork-Infused Camera Pose Localization and an Extended Cambridge Landmarks Dataset (CVPR 2025 · similarity 0.87)
- Neural Refinement for Absolute Pose Regression with Feature Synthesis (CVPR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.