Research map / Label-efficient and robust learning / Recognition and long-tail
M3LEO: A Multi-Modal, Multi-Label Earth Observation Dataset Integrating Interferometric SAR and Multispectral Data
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces M3LEO, a ~17M-chip multi-modal Earth observation dataset combining Sentinel-1 polarimetric amplitude, interferometry, coherence, Sentinel-2 and auxiliary terrain layers, with a PyTorch Lightning/Hydra framework and GEE tiling tools.
Problem
Full diversity of SAR data (polarimetry, coherence, interferometry) is not available at scale in ML-ready formats, and existing EO datasets are mostly optical, limiting ML research on SAR.
Approach
A large, spatially aligned, tiled multi-source dataset covering polarimetric, interferometric and coherence SAR plus Sentinel-2 and auxiliary labels across six AOIs, with geographic-band train/val/test splits and tooling (geetiles, sartiles) to convert any GEE dataset into the same tile format.
Key results
- S2RGB+S1GRD fusion achieved the best ESAWC mIoU of 0.4634 and AGB RMSE of 25.137 Mg/ha among the baseline UNet experiments.
- GSSIC coherence alone performed worst, with ESAWC mIoU 0.2906 and AGB RMSE 39.314 Mg/ha.
- Distribution shift in MAE embeddings across AOIs remained substantial even when conditioning on terrain properties (appearance shift).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Fuse S2RGB with S1GRD VV+VH instead of using either input alone | UNet · ESAWC | mIoU: 0.4185 -> 0.4634 | Helped |
| Fuse S2RGB with S1GRD VV+VH instead of using either input alone | UNet · AGB | RMSE: 27.467 -> 25.137 | Helped |
| Fuse S2RGB with S1GRD VV+VH instead of using either input alone | UNet · GHSBUILTS | RMSE: 131.968 -> 124.852 | Helped |
| Add VH to S1GRD VV input | UNet · ESAWC | mIoU: 0.3976 -> 0.4185 | Helped |
| Add VH to S1GRD VV input | UNet · AGB | RMSE: 28.573 -> 27.467 | Helped |
| Add VH to S1GRD VV input | UNet · GHSBUILTS | RMSE: 152.259 -> 141.719 | Helped |
| Add GSSIC coherence to S2RGB | UNet · ESAWC | mIoU: 0.4094 -> 0.4198 | Helped |
| Add GSSIC coherence to S2RGB | UNet · AGB | RMSE: 28.689 -> 28.550 | Helped |
| Add GSSIC coherence to S2RGB | UNet · GHSBUILTS | RMSE: 131.968 -> 131.300 | Helped |
| Use GSSIC coherence alone instead of S2RGB | UNet · ESAWC | mIoU: 0.4094 -> 0.2906 | Hurt |
| Use GSSIC coherence alone instead of S2RGB | UNet · AGB | RMSE: 28.689 -> 39.314 | Hurt |
| Use GSSIC coherence alone instead of S2RGB | UNet · GHSBUILTS | RMSE: 131.968 -> 196.242 | Hurt |
Surprising findings
- Conditioning embeddings on similar terrain properties did not substantially reduce cross-AOI embedding shift for the MAE model, unlike previously reported DINO behaviour on M3LEO.
- Coherence (GSSIC) as a naive direct input performed significantly worse than optical or amplitude SAR data.
Limitations
- Coverage is limited to regions with consistent Sentinel-1 IW acquisition modes and directional coverage, not global.
- Interferometry and coherence data are not used in the baseline experiments or change detection; time series data is limited to 2018-2020 and mostly 2020 for auxiliary datasets.
- No standardized benchmark or error bars; compute cost prevented repeated runs.
Experimental setup
Datasets: Sentinel-1 GRD (S1GRD) ASF ARIA GUNW interferograms GSSIC coherence Sentinel-2 surface reflectance (S2SRM) ESA WorldCover ESA CCI Biomass MODIS Vegetation Continuous Fields GHS Built Surface SRTM DEM
Benchmarks: ESA WorldCover segmentation (M3LEO) ESA CCI Biomass regression (M3LEO) GHS Built Surface regression (M3LEO)
Metrics: mIoU RMSE sliced Wasserstein distance Wasserstein distance L1 distance
Baselines: SSL4EO-S12 SSL4EO-L SatlasPretrain SEN12MS BigEarthNet SeCo
Builds on: MAE DINO CLIP Hydra PyTorch Lightning
Closest papers by meaning
- Kuro Siwo: 33 billion $m^2$ under the water. A global multi-temporal satellite dataset for rapid flood mapping (NeurIPS 2024 · similarity 0.85)
- SkySense: A Multi-Modal Remote Sensing Foundation Model Towards Universal Interpretation for Earth Observation Imagery (CVPR 2024 · similarity 0.85)
- Galileo: Learning Global & Local Features of Many Remote Sensing Modalities (ICML 2025 · similarity 0.84)
- SARDet-100K: Towards Open-Source Benchmark and ToolKit for Large-Scale SAR Object Detection (NeurIPS 2024 · similarity 0.83)
- RobSense: A Robust Multi-modal Foundation Model for Remote Sensing with Static, Temporal, and Incomplete Data Adaptability (CVPR 2025 · similarity 0.83)
- EarthDial: Turning Multi-sensory Earth Observations to Interactive Dialogues (CVPR 2025 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.