Research map / Image generation / Fast diffusion sampling
SatSynth: Augmenting Image-Mask Pairs through Diffusion Models for Aerial Semantic Segmentation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Trains a DDPM on the joint distribution of satellite images and binary-encoded label maps, samples new image-mask pairs, and uses them as augmentation for aerial semantic segmentation.
Problem
Semantic segmentation of satellite imagery requires costly expert pixel annotations, and conventional augmentations fail to capture the diversity of earth observation scenes.
Approach
Learn the joint image-label distribution with an unconditional DDPM by concatenating binary-encoded label channels with RGB, sample synthetic image-mask pairs, and add them to the real training set; a conditional DDPM super-resolves the pairs to 256x256.
Key results
- On iSAID, the synthetic-augmented FPN reaches 52.13 IoU vs 41.25 for SegDiff and 13.01 for SemGAN; FID on iSAID improves from 10.21 (SemGAN) to 8.66.
- Training PSPNet on iSAID with D plus synthetic pairs raises mIoU by 7.59% relative to real data alone (48.95 to 56.54).
- Augmentation with synthetic pairs improves segmentation consistently across resampling ratios R=1 to 5 on iSAID, LoveDA, and OpenEarthMap.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Train segmentation with generated image-mask pairs D′ added to original data D | FPN · iSAID | IoU | Helped |
| Train segmentation with generated image-mask pairs D′ added to original data D | FPN · LoveDA | mIoU | Helped |
| Train segmentation with generated image-mask pairs D′ added to original data D | FPN · OpenEarthMap | mIoU | Helped |
| Train on generated pairs D′ as well as original data D | PFSegNet · iSAID | IoU: 60.93±0.62 -> 63.71±0.21 | Helped |
| Train on generated pairs D′ as well as original data D | FarSeg · iSAID | IoU: 62.28±0.27 -> 62.95±0.25 | Helped |
| Train on generated pairs D′ as well as original data D | SegFormer · iSAID | IoU: 60.95±0.61 -> 62.13±0.25 | Helped |
| Train on generated pairs D′ as well as original data D | FPN · iSAID | IoU: 59.52± 0.19 -> 60.65±0.38 | Helped |
| Train on generated pairs D′ as well as original data D | PSPNet · iSAID | IoU: 48.95±2.91 -> 56.54±1.32 | Helped |
| Use binary label encoding instead of one-hot encoding (ablation) | SatSynth diffusion model · iSAID, R = 1 | IoU: 47.98 -> 51.00 | Helped |
| Use binary label encoding instead of one-hot encoding (ablation) | SatSynth diffusion model · iSAID, R = 3 | IoU: 49.38 -> 51.45 | Helped |
| Predict noise epsilon instead of the original sample x0 (ablation) | SatSynth diffusion model with binary labels · iSAID, R = 1 | IoU: 51.00 -> 51.11 | Helped |
| Predict noise epsilon instead of the original sample x0 (ablation) | SatSynth diffusion model with binary labels · iSAID, R = 3 | IoU: 51.45 -> 52.13 | Helped |
Surprising findings
- Predicting the noise epsilon gave better segmentation than predicting x0 in the ablation, especially at R=3.
Limitations
- Synthetic samples become increasingly correlated and redundant as more are drawn, giving diminishing returns.
- Generation is limited to 128x128 base resolution, requiring a separate super-resolution model for higher resolutions.
Experimental setup
Datasets: iSAID LoveDA OpenEarthMap
Benchmarks: iSAID LoveDA OpenEarthMap
Metrics: FID sFID IS mIoU IoU F1
Baselines: SemGAN SegDiff DDPM PFSegNet FarSeg SegFormer PSPNet FPN
Builds on: DDPM SegDiff SemGAN Analog Bits
Closest papers by meaning
- DiffusionSat: A Generative Foundation Model for Satellite Imagery (ICLR 2024 · similarity 0.89)
- Towards Open-Vocabulary Remote Sensing Image Semantic Segmentation (AAAI 2025 · similarity 0.87)
- GeoCore-9B: Towards Geo-Aware Generative Foundation Models in Earth Observation (NeurIPS 2026 · similarity 0.87)
- EmerDiff: Emerging Pixel-level Semantic Knowledge in Diffusion Models (ICLR 2024 · similarity 0.86)
- $\mathbf{D^{3}S^{2}}$: Diffusion-Guided Dataset Distillation for Semantic Segmentation (NeurIPS 2026 · similarity 0.86)
- SemStereo: Semantic-Constrained Stereo Matching Network for Remote Sensing (AAAI 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.