Research map / Vision-language and multimodal / Human and embodied modeling
EMVP: Embracing Visual Foundation Model for Visual Place Recognition with Centroid-Free Probing
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes EMVP, a PEFT pipeline for visual place recognition that pairs a centroid-free second-order probing stage (CFP) with dynamic power normalization in both probing and backbone recalibration on a frozen DINOv2 ViT.
Problem
Adapting visual foundation models to visual place recognition (VPR) is typically done with first-order probing or costly NetVLAD centroid initialization, which limits accuracy and flexibility, while full fine-tuning is expensive for mobile robots.
Approach
Reinterprets NetVLAD as a bilinear pooling and removes explicit semantic centroids via constant normalization (CFP), then adds learnable dynamic power normalization (DPN) in both the CFP probing stage and inserted backbone blocks (DPNR) for parameter-efficient fine-tuning.
Key results
- EMVP-L reaches 93.9% R@1 on MSLS Validation, 96.5% R@1 on Pitts250k-test and 94.6% R@1 on SPED.
- The sequential DPNR variant (EMVP-B) trains only 0.05M parameters versus 0.14M for PSRP/CFP-style probing, saving 64.3% of trainable parameters versus the SALAD-style baseline setup described in the paper.
- In ablations, CFP (with DPNC) improves MSLS Val R@1 from 90.3 (centroid-removed baseline) to 92.6 for ViT-B.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add centroid-free probing with softmax constant normalization and DPNC rather than centroid-removed bilinear probing alone (ablation) | ViT-B VPR probing · MSLS Val | Recall@1: 90.3 -> 92.6 | Helped |
| Add centroid-free probing with softmax constant normalization and DPNC rather than centroid-removed bilinear probing alone (ablation) | ViT-B VPR probing · NordLand | Recall@1: 56.5 -> 74.6 | Helped |
| Add centroid-free probing with softmax constant normalization and DPNC rather than centroid-removed bilinear probing alone (ablation) | ViT-B VPR probing · Pitts250k-test | Recall@1: 94.4 -> 95.2 | Helped |
| Add centroid-free probing with softmax constant normalization and DPNC rather than centroid-removed bilinear probing alone (ablation) | ViT-B VPR probing · SPED | Recall@1: 88.0 -> 92.1 | Helped |
| Add softmax constant normalization (ablation) | centroid-removed bilinear probing · NordLand | Recall@1: 56.5 -> 68.0 | Helped |
| Add sequential DPNR recalibration (ablation) | CFP + PSRP VPR model · NordLand | Recall@1: 73.0 -> 76.4 | Helped |
| Use sequential instead of parallel DPNR (ablation) | CFP VPR model · MSLS Val | Recall@1: 92.4 -> 93.2 | Helped |
| Use DPNR in the last 4 blocks rather than 8 blocks (ablation) | EMVP-B recalibration · MSLS Val | Recall@1: 89.2 -> 93.2 | Helped |
| Use the ViT-L instead of ViT-B backbone (ablation) | EMVP · MSLS Val | Recall@1: 93.2 -> 93.9 | Helped |
| Use the ViT-L instead of ViT-B backbone (ablation) | EMVP · Pitts250k-test | Recall@1: 95.7 -> 96.5 | Helped |
| Use CFP rather than linear probing (ablation) | ViT-B VPR model · NordLand | Recall@1: 38.1 -> 74.6 | Helped |
Surprising findings
- Simply removing NetVLAD centroids via bilinear pooling degrades performance, and constant normalization is needed to recover it.
- Zero-shot AnyLoc with DINOv2 lags far behind fine-tuned methods, indicating VFMs lack sufficient zero-shot VPR capability.
Limitations
- Only DINOv2 is studied as the visual foundation model; CLIP, SAM and others are not explored.
- Choice of recalibrated blocks and the CN implementation dependence are left to future work.
- Results rely on training on GSV-Cities and averaging over 5 runs, with generalization evaluated on a limited set of VPR datasets.
Experimental setup
Datasets: GSV-Cities MSLS Pittsburgh250k Pittsburgh30k Nordland SPED
Benchmarks: MSLS Validation Pittsburgh250k-test Pittsburgh30k-test Nordland SPED
Metrics: Recall@1 Recall@5 Recall@10 trainable parameters
Baselines: SALAD AnyLoc SelaVPR MixVPR NetVLAD MP LP GeM PSRP EigenPlaces TransVPR R2Former
Builds on: NetVLAD SALAD Moment Probing Matrix Power Normalization DINOv2
Closest papers by meaning
- EffoVPR: Effective Foundation Model Utilization for Visual Place Recognition (ICLR 2025 · similarity 0.90)
- SuperVLAD: Compact and Robust Image Descriptors for Visual Place Recognition (NeurIPS 2024 · similarity 0.87)
- Towards Seamless Adaptation of Pre-trained Models for Visual Place Recognition (ICLR 2024 · similarity 0.87)
- Towards Implicit Aggregation: Robust Image Representation for Place Recognition in the Transformer Era (NeurIPS 2025 · similarity 0.86)
- Focus on Local: Finding Reliable Discriminative Regions for Visual Place Recognition (AAAI 2025 · similarity 0.86)
- Optimal Transport Aggregation for Visual Place Recognition (CVPR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.