Research map / 3D vision / Depth and stereo
Learning to Produce Semi-dense Correspondences for Visual Localization
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes DeViLoc, a structure-based visual localization method that converts semi-dense 2D-2D matches into many 2D-3D matches via a point inference network (PIN) and confidence-based aggregation (CPA), trained only on MegaDepth.
Problem
Visual localization under night, weather and seasonal changes, where feature-matching methods discard unobserved keypoints and degrade with noisy or sparse 3D point clouds.
Approach
Regress depth/3D coordinates for all semi-dense 2D keypoints of the reference image from sparse observed 3D points and visual features using attention layers (PIN), then aggregate confidence-weighted 2D-3D matches across query-reference pairs (CPA), rather than using only pre-existing 3D points.
Key results
- On Extended CMU-Seasons Park, DeViLoc improves over HLoc+PixLoc by up to 5.1% recall; RobotCar night recall at (5m,10deg) 100% vs HLoc[SP+SG] 100%, with better night-time (0.25m,2deg) recall (87.8 vs 86.7).
- On 7scenes, DeViLoc ranks first in Fire, Heads and Pumpkin among methods and outperforms dense SCR methods despite being trained only on MegaDepth.
- On MegaDepth ablation, PIN+CPA raises AUC@5deg from 85.5 (SIFT 2D-3D matching, HLoc-style) to 90.5 with about 0.57s runtime.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add PIN to standard 2D-3D matching (ablation) | DeViLoc · MegaDepth | AUC at 2°: 72.4 -> 78.8 | Helped |
| add CPA to the PIN model (ablation) | DeViLoc · MegaDepth | AUC at 2°: 78.8 -> 79.6 | Helped |
| use SP+SG rather than SIFT point-cloud input (ablation) | DeViLoc · Aachen Day-Night Day | recall at 0.5m, 5°: 94.8 -> 95.3 | Helped |
| use SP+SG rather than SIFT point-cloud input (ablation) | DeViLoc · Aachen Day-Night Night | recall at 0.25m, 2°: 87.8 -> 88.8 | Helped |
| use LoFTR rather than SIFT point-cloud input (ablation) | DeViLoc · Aachen Day-Night Day | recall at 0.25m, 2°: 87.4 -> 87.9 | Helped |
| use half-SIFT rather than SIFT point-cloud input (ablation) | DeViLoc · Aachen Day-Night Night | recall at 0.25m, 2°: 87.8 -> 86.7 | Hurt |
| replace TopicFM with SP+SG (ablation) | DeViLoc · MegaDepth | AUC at 2°: 79.6 -> 78.2 | Hurt |
| retrieve top-5 rather than top-3 reference images (ablation) | DeViLoc · MegaDepth | AUC at 2°: 79.6 -> 81.0 | Helped |
| retrieve top-10 rather than top-3 reference images (ablation) | DeViLoc · MegaDepth | AUC at 2°: 79.6 -> 82.6 | Helped |
| use DeViLoc instead of HLoc+PixLoc | visual localization · Extended CMU-Seasons Park | recall at 25cm, 2°: 87.0 -> 92.1 | Helped |
Surprising findings
- Denser SP+SG or LoFTR point clouds only slightly improve DeViLoc, which works well even with noisy SIFT-based SfM inputs.
Limitations
- Runtime increases with the number of query-reference matching pairs.
- Performance gains are smaller with denser 3D inputs; generalization to scenes beyond the evaluated set not fully established.
Experimental setup
Datasets: MegaDepth 7scenes Cambridge Landmarks Aachen Day-Night RobotCar Seasons Extended CMU Seasons
Benchmarks: 7scenes Cambridge Landmarks Aachen Day-Night RobotCar Seasons Extended CMU Seasons MegaDepth
Metrics: median translation error (cm) median rotation error (deg) recall at (25cm,2deg)/(50cm,5deg)/(5m,10deg) AUC at 2/5/10 degrees
Baselines: HLoc[SP+SG] HLoc[LoFTR] HLoc[TopicFM] PixLoc NeuMap DSAC* DSAC++ HSCNet SANet DSM Active Search InLoc
Builds on: HLoc TopicFM depth completion NeuMap
Closest papers by meaning
- Few-Shot Geometry-Aware Keypoint Localization (CVPR 2023 · similarity 0.87)
- GLACE: Global Local Accelerated Coordinate Encoding (CVPR 2024 · similarity 0.85)
- HomoMatcher: Achieving Dense Feature Matching with Semi-Dense Efficiency by Homography Estimation (AAAI 2025 · similarity 0.85)
- GPVK-VL: Geometry-Preserving Virtual Keyframes for Visual Localization under Large Viewpoint Changes (CVPR 2025 · similarity 0.85)
- R-SCoRe: Revisiting Scene Coordinate Regression for Robust Large-Scale Visual Localization (CVPR 2025 · similarity 0.85)
- Fully Geometric Panoramic Localization (CVPR 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.