Research map / Label-efficient and robust learning / Recognition and long-tail
SatCLIP: Global, General-Purpose Location Embeddings with Satellite Imagery
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Pretrains a global location encoder (spherical harmonics + SIREN) by contrastively matching coordinates with frozen MoCo-pretrained Sentinel-2 image embeddings on 100K uniformly sampled tiles (S2-100K), and evaluates it on nine location-dependent tasks.
Problem
Location encoders for geospatial tasks often fail to generalize across geographic areas because they are trained on spatially biased data, and raw coordinates or full imagery are either poorly generalizable or expensive to use.
Approach
Train a general-purpose location encoder with the CLIP objective, matching coordinates to Sentinel-2 image embeddings from a frozen pretrained vision encoder over a globally, near-uniformly sampled dataset, using a spherical-harmonics plus SIREN location encoder.
Key results
- SatCLIP (L=40) achieves best scores on seven of nine downstream tasks in the interpolation setting, e.g., Countries 96.00% accuracy and Biome 94.41%.
- In zero/few-shot continent holdout, SatCLIP L=10 wins 8 of 14 comparisons (ViT16) versus 4 for GeoCLIP, the next best.
- Country classification in Africa reaches 30.65% accuracy with SatCLIP L=10 versus 0.45% for CSP-iNat and 0.48% for GPS2Vec.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use SatCLIP location embeddings instead of Identity coordinates | Downstream prediction · Air temperature | R²: 0.82 ± 0.16 -> 0.91 ± 0.01 | Helped |
| Use SatCLIP location embeddings instead of GeoCLIP | Downstream prediction · Countries | accuracy: 90.72 ± 0.44 -> 96.00 ± 0.14 | Helped |
| Use SatCLIP L=40 instead of L=10 (ablation) | SatCLIP · Biome | accuracy: 92.23 ± 0.26 -> 94.41 ± 0.14 | Helped |
| Use SatCLIP L=40 instead of L=10 (ablation) | SatCLIP · Cali. housing | R²: 0.35 ± 0.04 -> 0.57 ± 0.02 | Helped |
| Use SatCLIP L=40 rather than L=10 (ablation) | SatCLIP in geographic adaptation · Held-out continents | task wins: 8 vs. 5 task wins | Hurt |
| Change the image encoder among ViT-16, ResNet-18, and ResNet-50 (ablation) | SatCLIP · Spatial interpolation classification tasks | accuracy: <1% | No clear effect |
| Use SatCLIP embeddings instead of GeoCLIP | Downstream prediction · Median income | R²: 0.50 ± 0.01 vs. 0.47 ± 0.12 | Mixed |
Surprising findings
- Pure coordinate embeddings with spherical harmonics alone already produce biome-like T-SNE clusters, so part of the apparent structure comes from the encoding rather than learned weights.
- Several existing location encoders (CSP, GPS2Vec, GeoCLIP) perform worse than the raw-coordinate Identity baseline on some out-of-continent tasks.
Limitations
- Uses only Sentinel-2 imagery as contextual grounding; no other modalities such as audio or text.
- Fixed spatial scale set by L limits very fine-grained or local phenomena.
- No temporal dimension; time is marginalized over the two-year sampling window.
Experimental setup
Datasets: S2-100K iNaturalist 2018 MP-16 FMoW YFCC100M Air Temperature Elevation Median Income California Housing Population Density Biomes Ecoregions
Benchmarks: Air Temperature Elevation Median Income California Housing Population Density Countries iNaturalist Biome Ecoregions
Metrics: R^2 accuracy MSE
Baselines: GPS2Vec CSP GeoCLIP MOSAIKS Identity
Builds on: CLIP MoCo SatCLIP location encoder from Rußwurm et al. spherical harmonics SIREN GeoCLIP CSP GPS2Vec
Closest papers by meaning
- Vision-Language Reasoning for Geolocalization: A Reinforcement Learning Approach (AAAI 2026 · similarity 0.87)
- PIGEON: Predicting Image Geolocations (CVPR 2024 · similarity 0.87)
- GeoViS: Geospatially Rewarded Visual Search for Remote Sensing Visual Grounding (CVPR 2026 · similarity 0.87)
- TorchSpatial: A Location Encoding Framework and Benchmark for Spatial Representation Learning (NeurIPS 2024 · similarity 0.87)
- RANGE: Retrieval Augmented Neural Fields for Multi-Resolution Geo-Embeddings (CVPR 2025 · similarity 0.87)
- Towards Faithful Reasoning in Remote Sensing: A Perceptually-Grounded GeoSpatial Chain-of-Thought for Vision-Language Models (ICLR 2026 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.