Research map / 3D vision / 3D generation
GPVK-VL: Geometry-Preserving Virtual Keyframes for Visual Localization under Large Viewpoint Changes
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes GPVK-VL, which trains a 2D Gaussian splatting model with confidence-aware depth (Dust3r) and normal (DSINE) priors, extracts a mesh, and renders virtual keyframes to make HLoc-style visual localization robust to large viewpoint changes.
Problem
Structure-based visual localization fails when the query view differs greatly in viewpoint from the reference keyframe database (e.g., a corridor mapped from one direction), and existing novel view synthesis methods do not preserve geometry under such changes.
Approach
Add confidence-aware depth (from Dust3r) and confidence-aware normal (from DSINE, modeled as von Mises-Fisher) regularization to 2D Gaussian splatting, extract a TSDF mesh, and render a dense set of geometry-preserving virtual keyframes placed on a 2D occupancy map for localization.
Key results
- On the proposed large-viewpoint dataset, GPVK-VL reaches 59.4/64.4/64.4% success on Scene001 at (2°,5cm)/(4°,8cm)/(6°,10cm), versus 25.7/29.7/29.7% for HLoc, and 82.1/86.3/86.3% on Scene006.
- On ScanNet view synthesis, GPVK-VL achieves the lowest FDS (1.20) versus 1.60 for 2DGS-Mesh and 1.72 for Colmap, though PSNR (16.32) is comparable to DRGS (16.53) and not the best.
- On 7Scenes and Cambridge, GPVK-VL is competitive with HLoc and improves on some sequences (e.g., Cambridge Kings College 0.25m/0.3° vs HLoc 0.12m/0.2°, while on Old Hospital 0.21m/0.4° vs 0.14m/0.3°).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use geometry-preserving virtual keyframes | HLoc localization pipeline · Scene001 | success rate (2°/5cm, 4°/8cm, 6°/10cm): 36.6/39.6/39.6 -> 59.4/64.4/64.4 | Helped |
| use geometry-preserving virtual keyframes | HLoc localization pipeline · Scene002 | success rate (2°/5cm, 4°/8cm, 6°/10cm): 85.2/85.2/86.8 -> 89.5/91.2/91.2 | Helped |
| use geometry-preserving virtual keyframes | HLoc localization pipeline · Scene003 | success rate (2°/5cm, 4°/8cm, 6°/10cm): 86.5/89.5/92.5 -> 83.5/88.0/92.5 | Mixed |
| use geometry-preserving virtual keyframes | HLoc localization pipeline · Scene004 | success rate (2°/5cm, 4°/8cm, 6°/10cm): 44.4/46.7/48.1 -> 57.7/62.2/63.7 | Helped |
| use geometry-preserving virtual keyframes | HLoc localization pipeline · Scene005 | success rate (2°/5cm, 4°/8cm, 6°/10cm): 65.8/68.3/68.3 -> 76.9/80.3/82.0 | Helped |
| use geometry-preserving virtual keyframes | HLoc localization pipeline · Scene006 | success rate (2°/5cm, 4°/8cm, 6°/10cm): 65.7/69.8/69.8 -> 82.1/86.3/86.3 | Helped |
| use GPVK-VL view synthesis | view synthesis methods on ScanNet · ScanNet | FDS: 1.60 -> 1.20 | Helped |
| add normal prior (ablation) | 2DGS · ScanNet | FDS: 1.60 -> 1.52 | Helped |
| add normal prior (ablation) | 2DGS · ScanNet | GC-FMR (4/6/8 pixels): 52.5/57.8/62.5 -> 53.3/56.6/59.2 | Mixed |
| add confidence-aware normal prior (ablation) | 2DGS · ScanNet | FDS: 1.60 -> 1.43 | Helped |
| add confidence-aware normal prior (ablation) | 2DGS · ScanNet | GC-FMR (4/6/8 pixels): 52.5/57.8/62.5 -> 60.8/66.7/72.1 | Helped |
| add depth prior (ablation) | 2DGS · ScanNet | FDS: 1.60 -> 1.71 | Hurt |
Surprising findings
- Generic 2DGS-based and mesh-based virtual keyframes with monocular depth priors (DRGS-Mesh+VK) do not outperform the HLoc baseline on several scenes, showing that geometry priors need confidence weighting.
- On ScanNet, GPVK-VL does not show significant advantages under traditional image metrics (PSNR/SSIM), though it wins on feature-based metrics.
Limitations
- Mesh extraction limits scalability to large-scale maps.
- Virtual keyframe placement is restricted to a 2D plane, limiting use beyond mobile robots.
- Does not outperform HLoc on standard 7Scenes and Cambridge benchmarks in all sequences; traditional image metrics show no clear advantage.
Experimental setup
Datasets: ScanNet 7Scenes Cambridge Landmarks
Benchmarks: ScanNet 7Scenes Cambridge Landmarks Large-viewpoint localization dataset (6 indoor scenes)
Metrics: PSNR SSIM LPIPS Feature Descriptor Similarity GC-FMR localization success rate median translation and rotation error
Baselines: HLoc DFNet LENS CROSSFIRE InLoc Colmap MVSplat ZeroNVS DRGS 2DGS
Builds on: 2D Gaussian Splatting DSINE DUSt3R DRGS HLoc
Closest papers by meaning
- RELOCATE: A Simple Training-Free Baseline for Visual Query Localization Using Region-Based Representations (CVPR 2025 · similarity 0.87)
- Gaussian Splatting Feature Fields for (Privacy-Preserving) Visual Localization (CVPR 2025 · similarity 0.87)
- 360Loc: A Dataset and Benchmark for Omnidirectional Visual Localization with Cross-device Queries (CVPR 2024 · similarity 0.87)
- Fully Geometric Panoramic Localization (CVPR 2024 · similarity 0.86)
- Enhancing Visual Localization with Cross-Domain Image Generation (ICML 2025 · similarity 0.85)
- VA-GS: Enhancing the Geometric Representation of Gaussian Splatting via View Alignment (NeurIPS 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.