Research map / Vision-language and multimodal / Human and embodied modeling
Towards Seamless Adaptation of Pre-trained Models for Visual Place Recognition
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Adapts a frozen DINOv2 ViT-L/14 with lightweight global and local adapters, trained with a mutual-nearest-neighbor local loss, to yield global and dense local features for fast two-stage visual place recognition without spatial verification.
Problem
Pre-trained foundation models are not tuned for visual place recognition: their features attend to dynamic objects, full fine-tuning causes catastrophic forgetting, and they do not produce dense local features needed for re-ranking in two-stage VPR, which usually requires slow RANSAC verification.
Approach
A hybrid global-local adapter scheme on a frozen DINOv2: serial and parallel bottleneck adapters in each transformer block for global adaptation, plus up-convolution layers for dense local features, trained with a mutual-nearest-neighbor local feature loss so that match counts can be used directly as re-ranking scores without geometric verification.
Key results
- SelaVPR reaches 94.0 R@1 on Tokyo24/7 and 92.8 R@1 on Pitts30k-test, a 5.4% and 1.3% absolute gain over prior SOTA, and ranks 1st on the MSLS challenge leaderboard (73.5 R@1).
- Re-ranking runtime is 0.112 s per query on Pitts30k-test versus 3.018 s for TransVPR, about 3-4% of the total two-stage time.
- Tunable parameters are 53M versus 304M for full fine-tuning of DINOv2, and training uses 0.30 h for one epoch versus 0.62 h for full fine-tuning.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| adapt DINOv2 with global and local adapters and local-feature re-ranking | DINOv2 ViT-L/14 · Tokyo24/7 | R@1: 88.6 -> 94.0 | Helped |
| fully fine-tune DINOv2-GeM (ablation) | frozen DINOv2-GeM · Tokyo24/7 | R@1: 67.3 -> 65.7 | Hurt |
| fully fine-tune DINOv2-GeM (ablation) | frozen DINOv2-GeM · Pitts30k-test | R@1: 81.3 -> 85.3 | Helped |
| add global adapters while freezing the backbone (ablation) | DINOv2-GeM · MSLS-val | R@1: 44.7 -> 87.4 | Helped |
| add local adaptation and local loss for re-ranking (ablation) | frozen DINOv2-GeM · Tokyo24/7 | R@1: 67.3 -> 87.6 | Helped |
| combine global and local adaptation (ablation) | DINOv2 ViT-L/14 · Pitts30k-test | R@1: 81.3 -> 91.4 | Helped |
| combine global and local adaptation (ablation) | DINOv2 ViT-L/14 · MSLS-val | R@1: 44.7 -> 90.8 | Helped |
| use both serial and parallel adapters instead of only one adapter (ablation) | global adaptation · Pitts30k-test | R@1: 90.9 -> 91.4; 91.3 -> 91.4 | Helped |
| set adapter bottleneck ratio to 0.5 instead of 0.25 or 0.75 (ablation) | global adaptation · Pitts30k-test | R@1: 91.0 -> 91.4; 90.9 -> 91.4 | Helped |
| use the class token instead of GeM as the global feature with re-ranking (ablation) | SelaVPR · MSLS-val | R@1: 90.8 -> 83.5 | Hurt |
| use coarse patch tokens instead of dense local features for re-ranking (ablation) | SelaVPR · MSLS-val | R@1: 87.4 -> 82.8 | Hurt |
| re-rank the top-20 candidates instead of top-100 (ablation) | SelaVPR · Tokyo24/7 | R@1: 94.0 -> 93.0 | Hurt |
Surprising findings
- Full fine-tuning of DINOv2 hurts Tokyo24/7 performance relative to the frozen backbone (65.7 vs 67.3 R@1), which the authors attribute to loss of transferability.
- Re-ranking with coarse 16x16 patch tokens degrades MSLS-val R@1 below global retrieval (82.8 vs 87.4), showing that dense local features are needed.
Limitations
- Dense local feature extraction with ViT-L/14 is slower in extraction than TransVPR (0.027 s vs 0.008 s).
- Uses match counts without geometric verification, relying on the claim that false matches are rare; this is argued empirically rather than guaranteed.
- Some reported comparisons use results from StructVPR's paper because its code is unreleased.
Experimental setup
Datasets: MSLS Pitts30k Tokyo24/7 Nordland St. Lucia Pitts250k
Benchmarks: Tokyo24/7 MSLS-val MSLS-challenge Pitts30k-test Nordland-test St. Lucia Pitts250k-test
Metrics: Recall@1 Recall@5 Recall@10 runtime (s)
Baselines: NetVLAD SFRS CosPlace MixVPR SP-SuperGlue Patch-NetVLAD TransVPR StructVPR R2Former
Builds on: DINOv2 Vision Transformer NetVLAD Patch-NetVLAD AdaptFormer
Closest papers by meaning
- Focus on Local: Finding Reliable Discriminative Regions for Visual Place Recognition (AAAI 2025 · similarity 0.91)
- EffoVPR: Effective Foundation Model Utilization for Visual Place Recognition (ICLR 2025 · similarity 0.90)
- MutualVPR: A Mutual Learning Framework for Resolving Supervision Inconsistencies via Adaptive Clustering (NeurIPS 2025 · similarity 0.88)
- Towards Implicit Aggregation: Robust Image Representation for Place Recognition in the Transformer Era (NeurIPS 2025 · similarity 0.88)
- Optimal Transport Aggregation for Visual Place Recognition (CVPR 2024 · similarity 0.88)
- CricaVPR: Cross-image Correlation-aware Representation Learning for Visual Place Recognition (CVPR 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.