Research map / Label-efficient and robust learning / Self-supervised and contrastive
Galileo: Learning Global & Local Features of Many Remote Sensing Modalities
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Galileo is a self-supervised, highly multimodal ViT for remote sensing that combines global (deep-target, space-time masked) and local (shallow-target, unstructured masked) latent-prediction objectives, pretrained on a globally sampled multimodal dataset.
Problem
Learning shared representations of remote sensing data is hard because of the diversity of sensor modalities (optical, SAR, elevation, weather, etc.), input shapes (images, pixel time series), and object scales ranging from single-pixel boats to kilometre-scale glaciers.
Approach
Dual masked-modeling objectives: a global task predicting deep encoder representations under structured space-time masking, and a local task predicting shallow linear-projection targets under unstructured masking with a PatchDisc contrastive loss, averaged into one loss; flexible tokenization over space, time, and channel groups lets one model handle many modalities and input shapes.
Key results
- Galileo-Base achieves average rank 3.0 on image tasks and 1.8 on pixel time series, ranking first among compared RS models.
- Galileo outperforms AnySat by 10.8% on EuroSat (image classification).
- Ablation: the dual global+local objective gives 91.00% EuroSat kNN and 67.79 MADOS mIoU, more consistent than single-objective variants.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Adding deep targets with space-time masking for the global task (vs. unstructured random masking) (ablation) | global task · m-EuroSat kNN (ablation) | Top-1 Acc.: 89.50 vs 17.40 | Helped |
| Adding the PatchDisc_B batch-negatives loss to the global task (ablation) | global task alone · m-EuroSat kNN (ablation) | Top-1 Acc.: 89.50 vs 62.50 | Helped |
| Using only linear projection targets (depth 0) instead of per-modality depths in the global task (ablation) | global task · m-EuroSat kNN (ablation) | Top-1 Acc.: 86.90 vs 89.50 | Hurt |
| Using shallow linear targets (depth 0) instead of deep targets in the local task (ablation) | local task · MADOS linear probe mIoU (ablation) | mIoU: 71.48 vs 58.53 (depth 6) | Helped |
| Using unstructured random masking instead of space+time masking in the local task (ablation) | local task · MADOS linear probe mIoU (ablation) | mIoU: 71.48 vs 62.25 | Helped |
| Using the MSE loss instead of PatchDisc contrastive loss in the local task (ablation) | local task · MADOS linear probe mIoU (ablation) | mIoU: 65.34 vs 71.48 | Hurt |
| Combining global and local objectives (dual-objective) vs. local alone (ablation) | Galileo full algorithm · EuroSat kNN (ablation) | Top-1 Acc.: 91.00 vs 62.50 (global-only PatchDisc) | Helped |
| Sharing predictor parameters across global and local objectives (ablation) | Galileo full algorithm · MADOS linear probe mIoU (ablation) | mIoU: 67.04 vs 67.79 | Hurt |
| Removing TerraClimate input from pretraining data (ablation) | Galileo pretraining data · MADOS linear probe mIoU (ablation) | mIoU: 61.30 vs 67.79 | Hurt |
| Removing VIIRS nighttime lights input from pretraining data (ablation) | Galileo pretraining data · EuroSat kNN (ablation) | Top-1 Acc.: 81.10 vs 91.00 | Hurt |
| Removing ERA5 / NDVI / SRTM / LandScan / location inputs from pretraining data (ablation) | Galileo pretraining data · EuroSat kNN (ablation) | Top-1 Acc.: 89.3-91.2 vs 91.00 | No clear effect |
| Galileo-Base vs. CROMA and other image models on m-EuroSat kNN | Galileo ViT-Base · m-EuroSat 100% kNN | Top-1 Acc.: 93.0 vs 89.7 (CROMA) | Helped |
Surprising findings
- PatchDisc loss without the local task collapses to 62.5% on EuroSat, plausibly exploiting position-embedding shortcuts; batch-level negatives fix this for the global task alone.
- Unstructured random masking fails for the global task (17.4% on EuroSat) while structured space-time masking reaches 89.5%.
Limitations
- Gains on standard benchmarks are limited to optical/radar inputs, so the benefit of extra modalities is not measured by these benchmarks.
- Ablations use ViT-Tiny for only 200 epochs, and some ablation results are noisy across runs.
- AnySat per-pixel evaluation required special handling and was checked only via personal communication with the authors.
Experimental setup
Datasets: Galileo pretraining dataset (Sentinel-1/2, NDVI, SRTM, Dynamic World, WorldCereal, ERA5, TerraClimate, VIIRS, LandScan) m-EuroSat m-BigEarthNet m-So2Sat m-Brick-Kiln m-Cashew-Plant m-SA-Crop-Type MADOS Sen1Floods11 PASTIS BreizhCrops CropHarvest
Benchmarks: GeoBench (m-EuroSat, m-BigEarthNet, m-So2Sat, m-Brick-Kiln, m-Cashew-Plant, m-SA-Crop-Type) MADOS Sen1Floods11 PASTIS BreizhCrops CropHarvest
Metrics: top-1 accuracy F1 score mIoU average rank
Baselines: SatMAE SatMAE++ CROMA SoftCon DOFA Satlas MMEarth DeCUR Prithvi 2.0 AnySat Presto
Builds on: LatentMIM I-JEPA masked autoencoder EMA target encoder
Closest papers by meaning
- SkySense: A Multi-Modal Remote Sensing Foundation Model Towards Universal Interpretation for Earth Observation Imagery (CVPR 2024 · similarity 0.89)
- Bridging Remote Sensors with Multisensor Geospatial Foundation Models (CVPR 2024 · similarity 0.86)
- Parameter Efficient Self-Supervised Geospatial Domain Adaptation (CVPR 2024 · similarity 0.86)
- GeoLink: Empowering Remote Sensing Foundation Model with OpenStreetMap Data (NeurIPS 2025 · similarity 0.85)
- RoMA: Scaling up Mamba-based Foundation Models for Remote Sensing (NeurIPS 2025 · similarity 0.85)
- SkyMoE: A Vision-Language Foundation Model for Enhancing Geospatial Interpretation with Mixture of Experts (AAAI 2026 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.