Research map / 3D vision / Human pose estimation
The Unreasonable Effectiveness of Pre-Trained Features for Camera Pose Refinement
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes MCLoc, a training-free camera pose refiner that combines a particle filter with render-and-compare over dense pre-trained CNN features (coarse-to-fine across layers) on a renderable mesh or 3D Gaussian Splatting scene.
Problem
Camera pose refinement for visual localization: whether scene-specific trained features or implicit representations are necessary to refine an initial pose, or whether generic pre-trained features suffice.
Approach
Use dense, pixelwise normalized feature distances from off-the-shelf pre-trained CNNs (switching from deep to shallow layers during optimization) as the cost in a render-and-compare loop, optimized by a multi-beam particle filter over SE(3), with no per-scene or feature training.
Key results
- On Cambridge Landmarks, MCLoc (0.31/0.42 m/° King's College etc.) outperforms per-scene implicit feature refiners such as CROSSFIRE and NeFeS on most scenes, with no per-scene training.
- On Aachen Day-Night v1.1 starting from hloc poses, refinement raises night recall at the finest threshold from 71.7% to 73.8%.
- As a pre-processing step before MeshLoc with LoFTR (top-10), night recall at (25cm,2°) rises from 70.7% to 73.8%.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Refine retrieval-initialized poses with MCLoc | Aachen Day-Night v1.1 · Aachen Day-Night v1.1 Day | recall (5m, 10°): 75.5 -> 89.7 | Helped |
| Refine retrieval-initialized poses with MCLoc | Aachen Day-Night v1.1 · Aachen Day-Night v1.1 Night | recall (5m, 10°): 60.7 -> 86.9 | Helped |
| Refine hloc estimates with MCLoc | hloc · Aachen Day-Night v1.1 Day | recall (25cm, 2°): 87.4 -> 87.9 | Helped |
| Refine hloc estimates with MCLoc | hloc · Aachen Day-Night v1.1 Night | recall (25cm, 2°): 71.7 -> 73.8 | Helped |
| Use CosPlace-trained features instead of ImageNet features (ablation) | MCLoc · ShopFacade and OldHospital | median pose error: 12 / 0.45 -> 12 / 0.55; 39 / 0.73 -> 46 / 0.80 | Mixed |
| Use SimCLR features instead of ImageNet features (ablation) | MCLoc · ShopFacade and OldHospital | median pose error: 12 / 0.55 -> 18 / 0.62; 46 / 0.80 -> 50 / 0.83 | Mixed |
| Use ALIKED features instead of ImageNet features (ablation) | MCLoc · ShopFacade and OldHospital | median pose error: 12 / 0.55 -> 17 / 0.64; 46 / 0.80 -> 49 / 0.84 | Mixed |
| Use AlexNet features instead of ImageNet features (ablation) | MCLoc · ShopFacade and OldHospital | median pose error: 12 / 0.55 -> 15 / 0.74; 46 / 0.80 -> 53 / 0.88 | Mixed |
| Switch to shallower feature maps in later optimization (ablation) | MCLoc · pose refinement | pose refinement accuracy | Helped |
| Use DINOv2 features instead of ImageNet features (ablation) | MCLoc · 7 Scenes SFM | median pose error: 2 / 0.8 -> 3 / 0.9; 3 / 1.4 -> 4 / 1.8; 3 / 1.3 -> 3 / 1.5; 4 / 1.3 -> 6 / 1.4; 5 / 1.6 -> 7 / 2.1; 6 / 1.6 -> 8 / 1.8; | Mixed |
| Use RoMa features instead of ImageNet features (ablation) | MCLoc · 7 Scenes SFM | median pose error: 2 / 0.8 -> 2 / 0.7; 3 / 1.4 -> 3 / 1.2; 3 / 1.3 -> 2 / 1.0; 4 / 1.3 -> 3 / 1.1; 5 / 1.6 -> 4 / 1.0; 6 / 1.6 -> 5 / 1.4; | Helped |
Surprising findings
- Generic ImageNet/ALIKED/AlexNet features yield similarly convergent pose-refinement basins as trained place-recognition features, suggesting specialized training is not necessary.
- Replacing the ImageNet-style features with DINOv2 gives comparable precision on 7scenes, while RoMa features (trained for matching) perform better.
Limitations
- Performs worse on indoor scenes with textureless, repetitive surfaces (7scenes), requiring more iterations.
- Relies on a renderable scene representation (mesh or Gaussian splatting) and the rendering domain gap, though relative differences are preserved.
- Slower than pose regressors at inference and trades off accuracy at fine thresholds on Aachen Day queries versus PixLoc.
Experimental setup
Datasets: Aachen Day-Night v1.1 Cambridge Landmarks 7scenes
Benchmarks: Aachen Day-Night v1.1 Cambridge Landmarks 7scenes
Metrics: recall at (25cm,2°), (50cm,5°), (5m,10°) median translation error (m) median rotation error (°) median cm/° error
Baselines: PixLoc FQN CROSSFIRE NeFeS DFNet LENS MS-Transformer hloc AS DSAC* HACNet MeshLoc
Builds on: particle filter LPIPS 3D Gaussian Splatting MeshLoc PixLoc
Closest papers by meaning
- GeoReF: Geometric Alignment Across Shape Variation for Category-level Object Pose Refinement (CVPR 2024 · similarity 0.88)
- RefPose: Leveraging Reference Geometric Correspondences for Accurate 6D Pose Estimation of Unseen Objects (CVPR 2025 · similarity 0.85)
- Neural Refinement for Absolute Pose Regression with Feature Synthesis (CVPR 2024 · similarity 0.85)
- SynSP: Synergy of Smoothness and Precision in Pose Sequences Refinement (CVPR 2024 · similarity 0.85)
- Reloc3r: Large-Scale Training of Relative Camera Pose Regression for Generalizable, Fast, and Accurate Visual Localization (CVPR 2025 · similarity 0.84)
- Map-Relative Pose Regression for Visual Re-Localization (CVPR 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.