Research map / Vision-language and multimodal / CLIP and zero-shot
EffoVPR: Effective Foundation Model Utilization for Visual Place Recognition
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Shows that DINOv2 self-attention features give a zero-shot re-ranker for visual place recognition, and that fine-tuning only the last layers with a CLS-token classification loss yields compact, state-of-the-art global descriptors (EffoVPR).
Problem
Visual place recognition with foundation models: vanilla DINOv2 is considered inadequate for VPR without fine-tuning, and existing SOTA relies on large external pooling layers that produce high-dimensional descriptors.
Approach
Use DINOv2's internal self-attention (CLS-attention-filtered Value features from layer n-1, with mutual-nearest-neighbor counting) as a training-free local re-ranker, and train only the CLS token of the last five ViT layers with a classification loss (EigenPlaces-style cells, CosFace) so that the built-in attention performs implicit pooling, enabling compact features down to 128D.
Key results
- EffoVPR-R (two-stage) reaches 98.7% R@1 on Tokyo24/7 and 79.0% on MSLS-challenge, and 95.0% on Nordland, versus prior SOTA such as SALAD at 94.6%, 75.0% and 76.0%.
- Zero-shot EffoVPR-ZS reaches 89.4% R@1 on Pitts30k and 57.9% on Nordland, beating AnyLoc (87.7% and 16.1%) on those sets.
- Single-stage EffoVPR-G at 128D achieves 94.6% R@1 on Tokyo24/7, parity with SALAD's 8448D on that dataset.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use local Value features from layer n-1 for re-ranking instead of using global features alone | EffoVPR-R · Tokyo24/7 | Recall@1: 94.6% to 98.7% | Helped |
| use EffoVPR-ZS local-feature matching and re-ranking | DINOv2 zero-shot retrieval · Tokyo24/7 | Recall@1: 62.2 -> 90.8 | Helped |
| use EffoVPR-ZS local-feature matching and re-ranking | Anyloc zero-shot retrieval · Nordland | Recall@1: 16.1 -> 57.9 | Helped |
| use EffoVPR-R rather than the strongest listed baseline SALAD | VPR retrieval · SF-XL-Night | Recall@1: +15% | Helped |
| use EffoVPR-R rather than the strongest listed baseline SALAD | VPR retrieval · SVOX-Rain | Recall@1 | Mixed |
| extract re-ranking features from layer n-1 rather than omit re-ranking (ablation) | EffoVPR-R · MSLS-val | Recall@1: 90.9 -> 92.8 | Helped |
| extract re-ranking features from final layer n rather than layer n-1 (ablation) | EffoVPR-R · MSLS-val | Recall@1: 92.8 -> 88.2 | Hurt |
| use Value rather than Query local features (ablation) | EffoVPR-R · Tokyo24/7 | Recall@1: 96.5 -> 98.7 | Helped |
| add both attention-map threshold T1 and matching-score threshold T2 (ablation) | EffoVPR-R · Tokyo24/7 | Recall@1: 95.9 -> 98.7 | Helped |
| rerank top 5 candidates rather than top 100 (ablation) | EffoVPR-R · Tokyo24/7 | Recall@1: 98.7 -> 97.8 | Hurt |
| increase inference resolution from 224px to 504px (ablation) | EffoVPR-R · Tokyo24/7 | Recall@1: 98.4 -> 98.7 | Helped |
| fine-tune the entire backbone end-to-end rather than only the last five layers (ablation) | EffoVPR · VPR benchmarks | Recall@1 | Hurt |
Surprising findings
- Fine-tuning the entire DINOv2 backbone end-to-end drastically harms performance; training only the last five layers is best.
- Using the attention-selected Value features from layer n-1 works better than the final layer, which degrades results.
Limitations
- Zero-shot performance is bounded by first-stage Recall@100 (e.g., 81.5% on MSLS-Val), and the re-ranking is two-stage with added matching cost.
- Training uses SF-XL panoramas with heading information, which the method requires and which other training datasets do not provide.
- Failure case: visually identical objects (e.g., a cable car) in query and gallery cause incorrect zero-shot matches.
Experimental setup
Datasets: SF-XL
Benchmarks: Pitts30k Tokyo24/7 MSLS-val MSLS-challenge Nordland AmsterTime SF-XL-Occlusion SF-XL-Night SVOX SPED SF-R SF-XL-v1
Metrics: Recall@1 Recall@5 Recall@10
Baselines: AnyLoc SALAD BoQ CricaVPR SelaVPR EigenPlaces CosPlace MixVPR R2Former
Builds on: DINOv2 EigenPlaces CosFace AnyLoc
Closest papers by meaning
- Focus on Local: Finding Reliable Discriminative Regions for Visual Place Recognition (AAAI 2025 · similarity 0.91)
- Towards Seamless Adaptation of Pre-trained Models for Visual Place Recognition (ICLR 2024 · similarity 0.90)
- EMVP: Embracing Visual Foundation Model for Visual Place Recognition with Centroid-Free Probing (NeurIPS 2024 · similarity 0.90)
- MutualVPR: A Mutual Learning Framework for Resolving Supervision Inconsistencies via Adaptive Clustering (NeurIPS 2025 · similarity 0.88)
- Optimal Transport Aggregation for Visual Place Recognition (CVPR 2024 · similarity 0.88)
- Towards Implicit Aggregation: Robust Image Representation for Place Recognition in the Transformer Era (NeurIPS 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.