Research map / Architectures and efficiency / Attention and transformers
Towards Implicit Aggregation: Robust Image Representation for Place Recognition in the Transformer Era
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes ImAge, which prepends learnable aggregation tokens to patch tokens before a late transformer block so that the backbone's self-attention implicitly aggregates a global VPR descriptor, removing the explicit aggregator.
Problem
Visual place recognition relies on a backbone plus a separate aggregator (e.g., NetVLAD, SALAD, BoQ) to turn patch tokens into a global descriptor; this adds structural complexity and one-shot aggregation with no chance for refinement.
Approach
Add a small set of learnable aggregation tokens before a particular transformer block (at the frozen/trainable junction), let self-attention implicitly aggregate patch information into them over the subsequent blocks, and use the final aggregation tokens (flattened, L2-normalized) as the descriptor; initialize tokens with L2-normalized k-means cluster centers.
Key results
- ImAge achieves 94.5% R@1 on MSLS-val and 84.5% R@1 on MSLS-challenge, ranking 1st on the MSLS leaderboard, with 6144-dim descriptors.
- Under a consistent DINOv2-base-register / GSV-Cities setup, ImAge reaches 94.0% R@1 on Pitts30k versus 93.1% for BoQ and 92.5% for SALAD, with 0 aggregator parameters and 14.8 ms inference time.
- Inserting 8 aggregation tokens before the fourth-to-last block (fine-tuning only the last four blocks) gives the best trade-off; adding tokens before the first block raises training GPU memory to 34.00 GB versus 16.73 GB for ImAge.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use implicit aggregation tokens instead of an explicit aggregator | DINOv2-base-register VPR · Pitts30k, MSLS-val, Tokyo24/7, Nordland | R@1: Pitts30k: 93.1 -> 94.0; MSLS-val: 92.8 -> 93.0; Tokyo24/7: 95.6 -> 96.2; Nordland: 87.0 -> 93.2 | Mixed |
| Delay token insertion to the junction between frozen and trainable blocks (ablation) | ImAge · Pitts30k | R@1: 88.5 -> 94.0 | Helped |
| Train all blocks when inserting aggregation tokens before the first block (ablation) | ImAge · Pitts30k | R@1: 88.5 -> 92.6 | Helped |
| Insert aggregation tokens before the penultimate block rather than at the frozen/trainable junction (ablation) | ImAge · Pitts30k | R@1: 94.0 -> 93.2 | Hurt |
| Add aggregation tokens progressively across the last four blocks rather than at the junction (ablation) | ImAge · Pitts30k | R@1: 94.0 -> 93.3 | Hurt |
| Initialize aggregation tokens with a normal distribution instead of zeros (ablation) | ImAge · Pitts30k, MSLS-val, Nordland | R@1: Pitts30k: 92.1 -> 92.9; MSLS-val: 89.6 -> 92.0; Nordland: 68.9 -> 88.6 | Helped |
| Initialize aggregation tokens with k-means centers instead of a normal distribution (ablation) | ImAge · Pitts30k, MSLS-val, Nordland | R@1: Pitts30k: 92.9 -> 93.5; MSLS-val: 92.0 -> 92.6; Nordland: 88.6 -> 91.7 | Helped |
| L2-normalize k-means centers used to initialize aggregation tokens (ablation) | ImAge · Pitts30k, MSLS-val, Nordland | R@1: Pitts30k: 93.5 -> 94.0; MSLS-val: 92.6 -> 93.0; Nordland: 91.7 -> 93.2 | Helped |
| Use 8 aggregation tokens instead of 1 (ablation) | ImAge · Pitts30k, MSLS-val, Nordland | R@1: Pitts30k: 92.2 -> 94.0; MSLS-val: 90.7 -> 93.0; Nordland: 74.8 -> 93.2 | Helped |
| Increase aggregation token count from 8 to 16 (ablation) | ImAge · Pitts30k, MSLS-val, Nordland | R@1: Pitts30k: 94.0 -> 93.7; MSLS-val: 93.0 -> 92.8; Nordland: 93.2 -> 92.2 | Hurt |
| Increase aggregation token count from 8 to 64 (ablation) | ImAge · Pitts30k, MSLS-val, Nordland | R@1: Pitts30k: 94.0 -> 92.8; MSLS-val: 93.0 -> 92.2; Nordland: 93.2 -> 85.4 | Hurt |
| Replace the class token with 8 aggregation tokens (ablation) | ImAge · Pitts30k, MSLS-val, Nordland | R@1: Pitts30k: 91.8 -> 94.0; MSLS-val: 89.1 -> 93.0; Nordland: 63.5 -> 93.2 | Helped |
Surprising findings
- Adding aggregation tokens before the first block (with all blocks trainable) performs worse than inserting them at the frozen/trainable junction, despite having more trainable capacity.
- A single aggregation token already beats the class token baseline with the same dimensionality, with an 11.3% R@1 gain on Nordland.
Limitations
- Gains are smaller on backbones without large-scale pretraining (ViT pre-trained on ImageNet, CLIP) than with DINOv2.
- Not suited when the backbone must stay frozen (e.g., AnyLoc) or when the backbone is very expensive to fine-tune.
- Global-descriptor-only retrieval still fails on large viewpoint changes where local-feature re-ranking (SelaVPR) succeeds.
Experimental setup
Datasets: GSV-Cities Pitts30k MSLS SF-XL
Benchmarks: Pitts30k MSLS-val MSLS-challenge Tokyo24/7 Nordland Baidu Mall SPED Pitts250k St. Lucia SVOX
Metrics: Recall@1 Recall@5 Recall@10 inference time (ms) training GPU memory (GB)
Baselines: NetVLAD SALAD BoQ SALAD-CM EDTformer CricaVPR SuperVLAD SelaVPR MixVPR EigenPlaces CosPlace SFRS
Builds on: DINOv2-register NetVLAD Vision Transformer Multi-similarity loss Prompt tuning
Closest papers by meaning
- Optimal Transport Aggregation for Visual Place Recognition (CVPR 2024 · similarity 0.92)
- SuperVLAD: Compact and Robust Image Descriptors for Visual Place Recognition (NeurIPS 2024 · similarity 0.91)
- Focus on Local: Finding Reliable Discriminative Regions for Visual Place Recognition (AAAI 2025 · similarity 0.90)
- MutualVPR: A Mutual Learning Framework for Resolving Supervision Inconsistencies via Adaptive Clustering (NeurIPS 2025 · similarity 0.89)
- Towards Seamless Adaptation of Pre-trained Models for Visual Place Recognition (ICLR 2024 · similarity 0.88)
- EffoVPR: Effective Foundation Model Utilization for Visual Place Recognition (ICLR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.