Research map / Detection and segmentation / Open-set and few-shot detection
Benchmarking Object Detectors under Real-World Distribution Shifts in Satellite Imagery
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces RWDS, three domain-generalisation object detection benchmarks built from xView and xDB satellite imagery (climate zones, flooded regions, hurricane events), and benchmarks six detectors under single- and multi-source settings.
Problem
Object detectors trained under the i.i.d. assumption degrade substantially under real-world spatial distribution shifts in satellite imagery, and no standardised domain generalisation benchmark for object detection exists for such high-impact real-world contexts.
Approach
Builds three domain-split satellite object detection datasets (climate zones from xView; flooded regions and hurricane events from xDB) with leave-one-domain-out evaluation adapted to detection, and reports performance drop and harmonic mean metrics for SOTA detectors in single- and multi-source setups.
Key results
- On RWDS-CZ single-source, Grounding DINO attains the best harmonic mean across targets (e.g., 9.5 on CZ A, 10.4 on CZ B, 9.2 on CZ C), with GLIP showing the lowest performance drop.
- On RWDS-FR, OOD mAP drops exceed 52% for India and 62% for the US across detectors; Grounding DINO and GLIP reach US OOD mAP of 10.8 and 11.9.
- Multi-source training improves OOD mAP across most detectors and domains on RWDS-CZ, e.g., Grounding DINO CZ A OOD mAP rises from 7.5 to 8.8.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| train on multiple source climate zones instead of one source zone | Object detectors on RWDS-CZ · RWDS-CZ OOD targets CZ A, CZ B, and CZ C | mAPOOD | Helped |
| train on multiple source climate zones instead of one source zone | Grounding DINO · RWDS-CZ OOD target CZ A | mAPOOD: 7.5 -> 8.8 | Helped |
| train on multiple source climate zones instead of one source zone | GLIP · RWDS-CZ OOD target CZ B | mAPOOD: 8.2 -> 9.2 | Helped |
| train on multiple source climate zones instead of one source zone | Grounding DINO · RWDS-CZ OOD target CZ C | mAPOOD: 7.1 -> 9.3 | Helped |
| train on multiple source hurricane events instead of one event | Object detectors on RWDS-HE · RWDS-HE OOD targets Florence, Michael, Harvey, and Matthew | mAPOOD | Helped |
| train on multiple source hurricane events instead of one event | Grounding DINO · RWDS-HE OOD target Florence | mAPOOD: 17.4 -> 28.2 | Helped |
| train on multiple source hurricane events instead of one event | GLIP · RWDS-HE OOD target Florence | mAPOOD: 19.0 -> 30.7 | Helped |
| train on multiple source hurricane events instead of one event | Grounding DINO · RWDS-HE OOD target Matthew | mAPOOD: 1.2 -> 1.5 | Helped |
| train on multiple source hurricane events instead of one event | GLIP · RWDS-HE OOD target Matthew | mAPOOD: 1.3 -> 1.1 | Hurt |
| train on multiple source hurricane events instead of one event | Object detectors on RWDS-HE · RWDS-HE ID test sets | mAPID | Hurt |
| evaluate on an OOD climate zone rather than the ID source zone | Object detectors on RWDS-CZ · RWDS-CZ | mAP | Hurt |
| evaluate on a different flooded region rather than the ID source region | Object detectors on RWDS-FR · RWDS-FR | mAP | Hurt |
Surprising findings
- Multi-source training slightly reduces in-domain performance on RWDS-HE while improving OOD performance.
- Open-set grounding detectors (Grounding DINO, GLIP) show the strongest robustness, which the authors attribute to their open-set design.
Limitations
- Detectors use default hyperparameters without tuning, so results may not reflect best achievable performance.
- Absolute mAP values are low (roughly 1 to 40), and classes and domains are limited in scope (16 classes for CZ, binary labels for disaster datasets).
- Domain definitions are based on geographic or event splits, which may confound multiple factors such as labelling and class imbalance.
Experimental setup
Datasets: xView xDB COCO-O
Benchmarks: RWDS-CZ RWDS-FR RWDS-HE
Metrics: mAP (COCO AP) performance drop harmonic mean
Baselines: Faster R-CNN Mask R-CNN TOOD DINO Grounding DINO GLIP
Builds on: COCO-O xView xDB MMDetection SAHI
Closest papers by meaning
- COUNTS: Benchmarking Object Detectors and Multimodal Large Language Models under Distribution Shifts (CVPR 2025 · similarity 0.88)
- Improving Single Domain-Generalized Object Detection: A Focus on Diversification and Alignment (CVPR 2024 · similarity 0.87)
- PhysAug: A Physical-guided and Frequency-based Data Augmentation for Single-Domain Generalized Object Detection (AAAI 2025 · similarity 0.87)
- Unbiased Faster R-CNN for Single-source Domain Generalized Object Detection (CVPR 2024 · similarity 0.87)
- Locate Anything on Earth: Advancing Open-Vocabulary Object Detection for Remote Sensing Community (AAAI 2025 · similarity 0.87)
- Generalized Diffusion Detector: Mining Robust Features from Diffusion Models for Domain-Generalized Detection (CVPR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.