Research map / Image generation / Text-to-image diffusion
MMM-RS: A Multi-modal, Multi-GSD, Multi-scene Remote Sensing Dataset and Benchmark for Text-to-Image Generation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Builds MMM-RS, a ~2.1M text-image remote sensing dataset spanning RGB/SAR/NIR modalities, multiple GSD levels and synthetic weather scenes, and fine-tunes Stable Diffusion 1.5 with LoRA and ControlNet on it for text-to-image and cross-modal generation.
Problem
Off-the-shelf text-to-image models cannot generate diverse remote sensing images because no public RS dataset pairs multi-modal imagery with information-rich text, multiple GSD levels, and varied scenes/weather.
Approach
Standardize nine public RS datasets to 512x512, auto-caption with BLIP-2 plus GSD, weather and satellite metadata into structured prompts, derive multiple GSD versions from one large image by cropping at different sizes, and synthesize fog, snow and low-light scenes to create a multi-modal, multi-GSD, multi-scene text-image benchmark.
Key results
- Fine-tuned SD1.5 on MMM-RS reaches FID 92.33 versus 172.78 for SD1.5, 168.34 for SDXL and 347.88 for DALL-E 3 (500 generated images per model).
- Ours obtains IS 7.21, the highest among compared models (SDXL 6.89, SD1.5 6.64).
- MMM-RS contains 2,103,273 text-image pairs, with 1,806,889 RGB, 289,384 SAR and 7,000 NIR samples.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Fine-tune SD1.5 with LoRA on MMM-RS instead of using SDXL | Remote sensing image generation · Remote sensing image generation | FID: 168.34 -> 92.33 | Helped |
| Fine-tune SD1.5 with LoRA on MMM-RS instead of using SDXL | Remote sensing image generation · Remote sensing image generation | IS: 6.89 -> 7.21 | Helped |
| Fine-tune SD1.5 with LoRA on MMM-RS instead of using SD1.5 | Remote sensing image generation · Remote sensing image generation | FID: 172.78 -> 92.33 | Helped |
| Fine-tune SD1.5 with LoRA on MMM-RS instead of using SD1.5 | Remote sensing image generation · Remote sensing image generation | IS: 6.64 -> 7.21 | Helped |
| Fine-tune SD1.5 with LoRA on MMM-RS instead of using SD2.0 | Remote sensing image generation · Remote sensing image generation | FID: 175.68 -> 92.33 | Helped |
| Fine-tune SD1.5 with LoRA on MMM-RS instead of using SD2.0 | Remote sensing image generation · Remote sensing image generation | IS: 6.31 -> 7.21 | Helped |
| Fine-tune SD1.5 with LoRA on MMM-RS instead of using DALL-E 3 | Remote sensing image generation · Remote sensing image generation | FID: 347.88 -> 92.33 | Helped |
| Fine-tune SD1.5 with LoRA on MMM-RS instead of using DALL-E 3 | Remote sensing image generation · Remote sensing image generation | IS: 2.63 -> 7.21 | Helped |
| Use the MMM-RS-trained model for snow, fog, and night prompts rather than the other compared methods | Remote sensing image generation · Multi-scene RS image generation | weather-consistent visual generation | Helped |
| Use the MMM-RS-trained model with GSD prompts rather than the other compared models | Remote sensing image generation · Multi-GSD RS image generation | GSD variation fidelity | Helped |
Surprising findings
- DALL-E 3 scores far worse than open diffusion baselines on FID (347.88) and IS (2.63) for RS generation.
Limitations
- Fog, snow and low-light scenes are synthetic rather than real captures, and the snow model was trained on only 460 image pairs.
- Evaluation relies only on FID and IS over 500 generated images per model, with no human or downstream-task evaluation.
- Category distribution is dominated by fMoW (over half of samples), so coverage is skewed toward recreational facilities and crop fields.
Experimental setup
Datasets: MRSSC2.0 Inria NaSC-TG2 GID WHU-OPT-SAR HRSC2016 TGRS-HRRSD fMoW SEN1-2 RSICD UCM-Captions
Benchmarks: MMM-RS
Metrics: FID IS
Baselines: SD2.0 SDXL DALL-E 3
Builds on: Stable Diffusion ControlNet BLIP-2 ESRGAN CycleGAN
Closest papers by meaning
- DiffusionSat: A Generative Foundation Model for Satellite Imagery (ICLR 2024 · similarity 0.90)
- GeoLLaVA-8K: Scaling Remote-Sensing Multimodal Large Language Models to 8K Resolution (NeurIPS 2025 · similarity 0.88)
- SkyScript: A Large and Semantically Diverse Vision-Language Dataset for Remote Sensing (AAAI 2024 · similarity 0.88)
- GeoChat: Grounded Large Vision-Language Model for Remote Sensing (CVPR 2024 · similarity 0.87)
- EcoMapper: Generative Modeling for Climate-Aware Satellite Imagery (ICML 2025 · similarity 0.87)
- SkySense-O: Towards Open-World Remote Sensing Interpretation with Vision-Centric Visual-Language Modeling (CVPR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.