Research map / Label-efficient and robust learning / Recognition and long-tail
Optimal Transport Aggregation for Visual Place Recognition
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SALAD, an optimal-transport (Sinkhorn) reformulation of NetVLAD aggregation with a dustbin cluster, combined with a partially fine-tuned DINOv2 backbone for single-stage visual place recognition.
Problem
Visual place recognition requires compact, discriminative global descriptors robust to appearance and viewpoint changes; existing single-stage aggregation (NetVLAD, GeM, MixVPR) and pretrained backbones leave room for improvement, and strong results often rely on costly re-ranking.
Approach
Treat NetVLAD's feature-to-cluster soft assignment as an optimal transport problem solved by Sinkhorn iterations, enforcing both feature-to-cluster and cluster-to-feature marginals, add a learnable dustbin cluster to discard uninformative features, drop residual subtraction and priors, and fine-tune the last DINOv2 blocks end to end.
Key results
- DINOv2 SALAD reaches 75.0% R@1 on MSLS Challenge and 76.0% R@1 on NordLand, outperforming prior single-stage methods by +7.6 and +17.6 points respectively.
- Single-stage DINOv2 SALAD outperforms two-stage re-ranking methods (R2Former, TransVPR, Patch-NetVLAD) on all benchmarks while running at about 2.4 ms per image.
- On SF-XL, DINOv2 SALAD achieves 88.6 R@1 (test v1) vs 84.1 for EigenPlaces, despite never training on San Francisco images.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use DINOv2 SALAD instead of the strongest single-stage baseline | Visual place recognition · MSLS Challenge | Recall@1: 67.4 -> 75.0 | Helped |
| Use DINOv2 SALAD instead of the strongest single-stage baseline | Visual place recognition · NordLand | Recall@1: 58.4 -> 76.0 | Helped |
| Fine-tune the final 4 DINOv2 blocks rather than freezing the backbone (ablation) | DINOv2 SALAD · MSLS Val | Recall@1: 88.5 -> 92.2 | Helped |
| Fine-tune only the 2 last DINOv2 blocks (ablation) | DINOv2 SALAD · MSLS Val | Recall@1: 88.5 -> 92.0 | Helped |
| Fine-tune the 6 last DINOv2 blocks rather than freezing the backbone (ablation) | DINOv2 SALAD · MSLS Val | Recall@1: 88.5 -> 91.6 | Helped |
| Fine-tune all DINOv2 blocks rather than freezing the backbone (ablation) | DINOv2 SALAD · MSLS Val | Recall@1: 88.5 -> 89.2 | No clear effect |
| Add the dustbin (ablation) | DINOv2 SALAD · MSLS Val | Recall@1: 91.4 -> 92.2 | Helped |
| Add the global token (ablation) | DINOv2 SALAD · MSLS Val | Recall@1: 91.8 -> 92.2 | Helped |
| Replace Dual Softmax assignment with Sinkhorn assignment (ablation) | DINOv2 SALAD · MSLS Val | Recall@1: 91.9 -> 92.2 | Helped |
| Use DINOv2-L rather than DINOv2-B (ablation) | SALAD · MSLS Val | Recall@1: 92.2 -> 92.6 | Helped |
| Use DINOv2-G rather than DINOv2-B (ablation) | SALAD · MSLS Val | Recall@1: 92.2 -> 91.7 | Hurt |
Surprising findings
- Fine-tuning the full DINOv2 backbone hurts performance (89.2 R@1 on MSLS Val) relative to fine-tuning only the last 2-4 blocks (92.0-92.2).
- SALAD applied to ResNet and ConvNext backbones improves over ResNet NetVLAD but is significantly worse than with DINOv2, suggesting it suits high-resolution DINOv2 features.
Limitations
- DINOv2 backbone yields slower processing than ResNet-based methods.
- SALAD's effectiveness depends on high spatial resolution features and is less suited to coarser backbone features.
- Uses only the simplest optimal transport assignment; more sophisticated constraints are left for future work.
Experimental setup
Datasets: GSV-Cities Pittsburgh30k-test MSLS NordLand Pitts250k-test SPED SF-XL
Benchmarks: MSLS Challenge MSLS Val NordLand Pitts250k-test SPED SF-XL
Metrics: Recall@1 Recall@5 Recall@10 latency (ms) memory (GB)
Baselines: NetVLAD GeM Conv-AP CosPlace MixVPR EigenPlaces Patch-NetVLAD TransVPR R2Former AnyLoc
Builds on: NetVLAD DINOv2 SuperGlue AnyLoc MixVPR
Closest papers by meaning
- Towards Implicit Aggregation: Robust Image Representation for Place Recognition in the Transformer Era (NeurIPS 2025 · similarity 0.92)
- SuperVLAD: Compact and Robust Image Descriptors for Visual Place Recognition (NeurIPS 2024 · similarity 0.91)
- Focus on Local: Finding Reliable Discriminative Regions for Visual Place Recognition (AAAI 2025 · similarity 0.90)
- MutualVPR: A Mutual Learning Framework for Resolving Supervision Inconsistencies via Adaptive Clustering (NeurIPS 2025 · similarity 0.89)
- EffoVPR: Effective Foundation Model Utilization for Visual Place Recognition (ICLR 2025 · similarity 0.88)
- Towards Seamless Adaptation of Pre-trained Models for Visual Place Recognition (ICLR 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.