Research map / 3D vision / 3D generation
SynRS3D: A Synthetic Dataset for Global 3D Semantic Understanding from Monocular Remote Sensing Imagery
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces SynRS3D, a 69,667-image procedurally generated synthetic remote sensing dataset with land cover, height, and building change labels, and RS3DAda, a self-training multi-task UDA method for synthetic-to-real land cover mapping and height estimation.
Problem
Monocular 3D semantic understanding from high-resolution remote sensing imagery is limited by costly annotations and geographically skewed real datasets, and synthetic-trained models suffer a large domain gap when transferred to real scenes.
Approach
Build a large procedural Blender-based synthetic RS dataset with nDSM height and land cover labels, then adapt to real imagery with a multi-task self-training teacher-student UDA where ground-class predictions mask height pseudo-labels, height-consistency filtering under perturbation, and a frozen DINOv2 feature constraint.
Key results
- RS3DAda reaches 48.23 mIoU on OEM land cover versus 40.70 source-only and 46.29 for DAFormer.
- On Target Domain 2 height estimation, RS3DAda outperforms models trained on real data, per the authors.
- Source-only SynRS3D achieves the best mIoU on JAX (67.89), OMA (62.44), Vaihingen (74.61), and Potsdam (59.69) among compared synthetic datasets.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use SynRS3D instead of SMARS or SyntheWorld for source-only training | Source-only land cover mapping · JAX | mIoU: 64.52 -> 67.89 | Helped |
| Use SynRS3D instead of SMARS or SyntheWorld for source-only training | Source-only land cover mapping · OMA | mIoU: 61.23 -> 62.44 | Helped |
| Use SynRS3D instead of SMARS or SyntheWorld for source-only training | Source-only land cover mapping · Vaihingen | mIoU: 72.15 -> 74.61 | Helped |
| Use SynRS3D instead of SMARS or SyntheWorld for source-only training | Source-only land cover mapping · Potsdam | mIoU: 52.49 -> 59.69 | Helped |
| Apply RS3DAda instead of source-only training | Land cover mapping on OEM · OEM | mIoU: 40.70 -> 48.23 | Helped |
| Add the ground mask to height pseudo-label refinement (ablation) | RS3DAda · Target Domain 2 | MAE: 6.117 to 5.652 | Helped |
| Add the ground mask to height pseudo-label refinement (ablation) | RS3DAda · Target Domain 2 | F1HE: 0.365 to 0.423 | Helped |
| Add height-map consistency filtering (ablation) | RS3DAda · Target Domain 2 | MAE: 5.652 to 5.253 | Helped |
| Add height-map consistency filtering (ablation) | RS3DAda · Target Domain 2 | F1HE: 0.423 to 0.425 | Helped |
| Add the feature constraint (ablation) | RS3DAda · Target Domain 2 | height estimation performance | Helped |
| Combine the ground mask, height consistency, and feature constraint (ablation) | RS3DAda · Target Domain 2 | MAE: 4.886 | Helped |
| Combine the ground mask, height consistency, and feature constraint (ablation) | RS3DAda · Target Domain 2 | F1HE: 0.485 | Helped |
Surprising findings
- Height-branch pseudo-label refinement via ground mask is reported as particularly crucial for height estimation performance.
Limitations
- Remaining synthetic-to-real appearance gap; RS3DAda still trails real-data-trained models on Target Domain 1.
- RS3DAda is specific to remote sensing synthetic-to-real multi-task UDA and not evaluated for other domain adaptation settings.
- Land cover mapping remains well below the Train-on-OEM oracle (48.23 vs 68.34 mIoU).
Experimental setup
Datasets: SynRS3D Houston JAX OMA GeoNRW Potsdam ATL ARG Nagoya Tokyo Vaihingen OEM
Benchmarks: JAX OMA Vaihingen Potsdam OEM Houston GeoNRW ATL ARG Nagoya Tokyo
Metrics: IoU mIoU MAE RMSE delta accuracy F1^HE
Baselines: DAFormer SMARS SyntheWorld Source-only Train-on-OEM
Builds on: self-training mean teacher ClassMix DPT DINOv2
Closest papers by meaning
- USCILab3D: A Large-scale, Long-term, Semantically Annotated Outdoor Dataset (NeurIPS 2024 · similarity 0.88)
- MMM-RS: A Multi-modal, Multi-GSD, Multi-scene Remote Sensing Dataset and Benchmark for Text-to-Image Generation (NeurIPS 2024 · similarity 0.87)
- GeoLLaVA-8K: Scaling Remote-Sensing Multimodal Large Language Models to 8K Resolution (NeurIPS 2025 · similarity 0.86)
- Learnable Earth Parser: Discovering 3D Prototypes in Aerial Scans (CVPR 2024 · similarity 0.86)
- SemStereo: Semantic-Constrained Stereo Matching Network for Remote Sensing (AAAI 2025 · similarity 0.86)
- SkyScript: A Large and Semantically Diverse Vision-Language Dataset for Remote Sensing (AAAI 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.