Research map / Image generation / Text-to-image diffusion
EcoMapper: Generative Modeling for Climate-Aware Satellite Imagery
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces the EcoMapper dataset of 2.9M Sentinel-2 RGB images paired with NASA POWER climate data across 15 land cover types, and fine-tunes Stable Diffusion 3 (text-to-image with climate prompts, and ControlNet multi-conditional generation) for climate-aware satellite image synthesis.
Problem
Satellite imagery lacks integration with climate data, limiting location-specific generative forecasting and climate scenario analysis; existing satellite generation models do not condition on future or observed climate conditions.
Approach
Fine-tune Stable Diffusion 3 with structured text prompts containing monthly temperature, precipitation and solar radiation, and add a ControlNet branch conditioned on prior-month satellite images with LoRA, so generated imagery reflects both climate and spatial structure, enabling time-series generation.
Key results
- SD3-FT-HR (1024x1024) achieved the best FID of 49.48, while SD3-FT reached CLIP 0.33 and SSIM 0.39 and PSNR 14.24.
- SD3 ControlNet multi-conditional model reached FID 48.20, SSIM 0.40, PSNR 13.63, LPIPS 0.59, outperforming text-only variants.
- Under extreme weather, SD3-FT FID rose to 170.80 for high precipitation versus 85.62 for low precipitation; grasslands FID 121.93 vs wetlands 263.36.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Fine-tune SD3 with climate metadata | SD3 · Global test set | FID: 157.36 -> 77.9 | Helped |
| Fine-tune DiffusionSat with climate metadata | DiffusionSat · Global test set | FID: 115.00 -> 59.8 | Helped |
| Use satellite-specific DiffusionSat pretraining instead of general-purpose SD3 | Base text-to-image models · Global test set | FID: 157.36 -> 115.00 | Helped |
| Fine-tune SD3 at 1024×1024 instead of 512×512 (ablation) | SD3-FT · Global test set | FID: 77.9 -> 49.48 | Helped |
| Fine-tune SD3 at 1024×1024 instead of 512×512 (ablation) | SD3-FT · Global test set | SSIM: 0.39 -> 0.36 | Hurt |
Surprising findings
- Generated images show minimal adaptation to extreme climate inputs in arid regions (e.g., snowy conditions in the UAE), and snowy rainforests are out-of-distribution generations.
- Although DiffusionSat was not trained on climate data, it outperformed the general-purpose SD3 base model without fine-tuning (FID 115.00 vs 157.36).
Limitations
- Generated images show weak climate responsiveness in regions where prompted climate deviates strongly from typical regional conditions.
- Performance is lower for underrepresented or visually complex classes such as wetlands and urban areas.
- Only RGB imagery and monthly climate inputs are used; no multispectral generation or higher-frequency climate data.
Experimental setup
Datasets: EcoMapper
Benchmarks: EcoMapper test set (5,500 locations, 2017-2024)
Metrics: FID CLIP score SSIM PSNR LPIPS
Baselines: DiffusionSat SD3 (base) DiffusionSat-FT
Builds on: Stable Diffusion 3 ControlNet DiffusionSat
Closest papers by meaning
- DiffusionSat: A Generative Foundation Model for Satellite Imagery (ICLR 2024 · similarity 0.89)
- MMM-RS: A Multi-modal, Multi-GSD, Multi-scene Remote Sensing Dataset and Benchmark for Text-to-Image Generation (NeurIPS 2024 · similarity 0.87)
- Multi-modal Learning for Geospatial Vegetation Forecasting (CVPR 2024 · similarity 0.87)
- GeoCore-9B: Towards Geo-Aware Generative Foundation Models in Earth Observation (NeurIPS 2026 · similarity 0.84)
- AnySat: One Earth Observation Model for Many Resolutions, Scales, and Modalities (CVPR 2025 · similarity 0.84)
- SynRS3D: A Synthetic Dataset for Global 3D Semantic Understanding from Monocular Remote Sensing Imagery (NeurIPS 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.