Research map / Label-efficient and robust learning / Self-supervised and contrastive
AnySat: One Earth Observation Model for Many Resolutions, Scales, and Modalities
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes AnySat, a JEPA-based self-supervised Earth observation model with scale-adaptive patch encoders, pretrained jointly on five heterogeneous datasets (GeoPlex) and evaluated by fine-tuning or linear probing on 9 downstream tasks.
Problem
Existing Earth observation foundation models expect fixed input configurations (resolution, scale, modality, channels), so they cannot be trained jointly on or applied to heterogeneous EO datasets without retraining.
Approach
Combine a multimodal JEPA (latent feature prediction, no modality-specific decoders) with a scale-adaptive patch encoder that uses fixed-size sub-patches so that different patch sizes and resolutions share one network, plus an InfoNCE cross-modal consistency loss, trained on several datasets simultaneously.
Key results
- Sets new state of the art on TreeSatAI-TS (+0.9 weighted F1) and PASTIS-HD (+2.8 mIoU classification, +0.2 segmentation) after fine-tuning.
- Outperforms prior methods on external datasets: SICKLE +3.6 mIoU, BraDD-S1TS +10.2 mIoU, TimeSen2Crop +11.0 OA.
- A linearly probed AnySat outperforms a fine-tuned Prithvi-v2 on Sen1Floods11 with 10^5 fewer free parameters.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| pretrain AnySat on GeoPlex | AnySat · TreeSatAI-TS test set | weighted F1: +0.9 weighted F1 score | Helped |
| pretrain AnySat on GeoPlex | AnySat · PASTIS-HD test set classification | mIoU: +2.8 mIoU | Helped |
| pretrain AnySat on GeoPlex | AnySat · PASTIS-HD test set semantic segmentation | mIoU: +0.2 | Helped |
| pretrain AnySat on GeoPlex | AnySat · SICKLE | mIoU: +3.6 mIoU | Helped |
| pretrain AnySat on GeoPlex | AnySat · BraDD-S1TS | mIoU: +10.2 mIoU | Helped |
| pretrain AnySat on GeoPlex | AnySat · TimeSen2Crop | OA: +11.0 OA | Helped |
| replace block masking with random token dropping (ablation) | AnySat · PASTIS-HD Fold 1 classification | macro F1: 72.0 -> 71.3 | Hurt |
| replace block masking with random token dropping (ablation) | AnySat · PASTIS-HD Fold 1 semantic segmentation | mIoU: 63.6 -> 64.1 | Helped |
| add contrastive loss (ablation) | AnySat · PASTIS-HD Fold 1 classification | macro F1: 67.7 -> 72.0 | Helped |
| add contrastive loss (ablation) | AnySat · PASTIS-HD Fold 1 semantic segmentation | mIoU: 63.4 -> 63.6 | Helped |
| use subpatch features rather than predicting logits directly from patch embeddings (ablation) | AnySat · PASTIS-HD Fold 1 semantic segmentation | mIoU: 61.2 -> 63.6 | Helped |
Surprising findings
- Block masking from JEPA is not critical for EO data; random token dropping decreased classification but slightly improved segmentation.
- Near state-of-the-art on PLANTED despite ViViT baselines trained on a withheld dataset with nearly 80% more data.
Limitations
- Some GeoPlex datasets are only partially public (PLANTED: 1.3 of 2.3M images), limiting reproducibility of the training data.
- Probing cannot be used for sensors too different from the training mix; new sensors require training a new projector.
Experimental setup
Datasets: TreeSatAI-TS FLAIR PLANTED S2NAIP-URBAN PASTIS-HD SICKLE BraDD-S1TS TimeSen2Crop Sen1Floods11 So2Sat HLS Burn Scar
Benchmarks: TreeSatAI-TS PASTIS-HD PLANTED FLAIR SICKLE BraDD-S1TS TimeSen2Crop Sen1Floods11 So2Sat HLS Burn Scar
Metrics: weighted F1 macro F1 mIoU IoU overall accuracy parameter count inference time
Baselines: OmniSat DOFA CROMA ScaleMAE SatMAE Prithvi Prithvi-v2 UTAE UT&T ViViT TSViT SatlasNet
Builds on: JEPA (I-JEPA) OmniSat DOFA ScaleMAE LTAE
Closest papers by meaning
- DUNIA: Pixel-Sized Embeddings via Cross-Modal Alignment for Earth Observation Applications (ICML 2025 · similarity 0.86)
- EarthDial: Turning Multi-sensory Earth Observations to Interactive Dialogues (CVPR 2025 · similarity 0.86)
- DiffusionSat: A Generative Foundation Model for Satellite Imagery (ICLR 2024 · similarity 0.86)
- GeoLLaVA-8K: Scaling Remote-Sensing Multimodal Large Language Models to 8K Resolution (NeurIPS 2025 · similarity 0.85)
- SkySense: A Multi-Modal Remote Sensing Foundation Model Towards Universal Interpretation for Earth Observation Imagery (CVPR 2024 · similarity 0.85)
- GeoLink: Empowering Remote Sensing Foundation Model with OpenStreetMap Data (NeurIPS 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.