Research map / Vision-language and multimodal / Human and embodied modeling
OpenStreetView-5M: The Many Roads to Global Visual Geolocation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces OpenStreetView-5M, a 5.1M-image open-access global street-view dataset with strict train/test spatial and sequence separation, and benchmarks encoders, prediction heads, fine-tuning and contrastive losses for visual geolocation.
Problem
Visual geolocation of images anywhere on Earth lacks large, open, reliably localizable benchmark datasets; existing web-scraped sets are noisy and biased, and street-view sets are proprietary or small, and train/test leakage can reduce geolocation to place recognition.
Approach
Build an open dataset from crowd-sourced Mapillary street-view images with enforced 1 km spatial separation and sequence separation between train and test, then systematically benchmark a modular geolocation framework (image encoder, QuadTree/hybrid/hierarchical heads, fine-tuning, region-contrastive loss) on it.
Key results
- Combined model (ViT-L-14 DATA COMP, QuadTree hybrid, hierarchical supervision, full fine-tuning, region-contrastive loss) reaches Geoscore 3361 and mean distance 1814 km, with country accuracy 68.0%, vs. base model 2052 and 2992 km.
- Human annotators score Geoscore 1009 with 6407 km distance on a 50-image subset, well below the combined model.
- Nearest-neighbor retrieval with StreetCLIP features achieves Geoscore 3597 and 1386 km distance, outperforming the combined model's retrieval (2734) since fine-tuning hurt retrieval.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use the combined model configuration instead of the base model | OSV-5M geolocation model · OSV-5M test | Geoscore: +1309 | Helped |
| Replace direct regression with a QuadTree classifier (ablation) | OSV-5M prediction head · OSV-5M test | Geoscore: + 720 | Helped |
| Use hybrid cell classification and relative-coordinate regression (ablation) | OSV-5M prediction head · OSV-5M test | Geoscore: + 264 | Helped |
| Add hierarchical supervision (ablation) | OSV-5M prediction head · OSV-5M test | Geoscore: + 118 | Helped |
| Fine-tune all image encoder parameters instead of freezing the encoder (ablation) | OSV-5M model · OSV-5M test | Geoscore: + 841 | Helped |
| Add region-based geographic contrastive learning (ablation) | Fine-tuned geolocation model · OSV-5M test | Geoscore: + 135 | Helped |
| Add area-based geographic contrastive pairs (ablation) | Fine-tuned geolocation model · OSV-5M test | Geoscore: 2893 -> 2376 | Hurt |
| Use text-based location descriptions for contrastive pairs (ablation) | Fine-tuned geolocation model · OSV-5M test | Geoscore: 2893 -> 2812 | Hurt |
| Replace coordinate regression with sine/cosine regression (ablation) | OSV-5M prediction head · OSV-5M test | Geoscore: 2052 -> 1192 | Hurt |
| Use a larger image encoder, ViT-bigG-14 rather than ViT-B-32 (ablation) | OSV-5M image encoder · OSV-5M test | Geoscore: 2052 -> 2878 | Helped |
| Use region-based rather than area-based contrastive pairs (ablation) | Fine-tuned geolocation model · OSV-5M test | Geoscore: 3028 vs. 2376 | Mixed |
Surprising findings
- Fine-tuning the image encoder for geolocation lowers nearest-neighbor retrieval performance, while the frozen pretrained encoder retrieves better.
- Using text descriptions of locations as contrastive positives is not beneficial, and area-level contrastive pairs hurt performance.
Limitations
- About 3.9% of images are estimated to be non-localizable, and the test-set localizability is only manually checked on 4500 images.
- Image density correlates only weakly with error (Pearson -0.25), so sparse coverage is not the sole cause of mistakes; South America shows higher error.
- Human evaluation uses only 80 annotators on 50 images.
Experimental setup
Datasets: OpenStreetView-5M Mapillary street view images LAION-2B DATA COMP ImageNet-1k
Benchmarks: OpenStreetView-5M test set
Metrics: Geoscore Haversine distance country/region/area/city classification accuracy
Baselines: StreetCLIP zero-shot GeoEstimator Random Human annotators Nearest neighbor retrieval
Builds on: CLIP StreetCLIP GeoEstimator LoRA MIL-NCE
Closest papers by meaning
- GeoRanker: Distance-Aware Ranking for Worldwide Image Geolocalization (NeurIPS 2025 · similarity 0.88)
- Around the World in 80 Timesteps: A Generative Approach to Global Visual Geolocation (CVPR 2025 · similarity 0.88)
- PIGEON: Predicting Image Geolocations (CVPR 2024 · similarity 0.88)
- GeoRouter: Dynamic Paradigm Routing for Worldwide Image Geolocalization (NeurIPS 2026 · similarity 0.87)
- GeoViS: Geospatially Rewarded Visual Search for Remote Sensing Visual Grounding (CVPR 2026 · similarity 0.87)
- Recognition through Reasoning: Reinforcing Image Geo-localization with Large Vision-Language Models (NeurIPS 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.