Research map / Detection and segmentation / Open-set and few-shot detection
UV-SAM: Adapting Segment Anything Model for Urban Village Identification
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Adapts the Segment Anything Model to urban village boundary segmentation from satellite images, using a lightweight SegFormer specialist to generate mask, box, and embedding prompts for SAM.
Problem
Existing urban village identification methods from satellite images either only classify images or produce inaccurate boundaries, and labeled data is scarce.
Approach
A generalist-specialist framework where a small trainable SegFormer produces coarse urban village masks, boxes, and image embeddings that are mixed into a prompt vector and fed to a frozen SAM mask decoder for fine-grained boundary segmentation.
Key results
- UV-SAM reaches IoU 0.721 on Beijing and 0.747 on Xi'an, outperforming the best baseline (FCN, 0.660 and 0.720) by roughly 4-9% on IoU.
- Human study: UV-SAM average mask rating 7.59 vs SegFormer 6.87 on Beijing (ground truth 7.67).
- Removing the box prompt drops Xi'an IoU from 0.747 to 0.173.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use UV-SAM instead of FCN | Urban village segmentation · Beijing | IoU: 0.660 -> 0.721 | Helped |
| use UV-SAM instead of FCN | Urban village segmentation · Beijing | F1-Score: 0.802 -> 0.871 | Helped |
| use UV-SAM instead of FCN | Urban village segmentation · Xi'an | IoU: 0.720 -> 0.747 | Helped |
| use UV-SAM instead of UVLens | Urban village segmentation · Xi'an | F1-Score: 0.863 -> 0.904 | Helped |
| add box prompt (ablation) | UV-SAM · Beijing | IoU: 0.708 -> 0.721 | Helped |
| add box prompt (ablation) | UV-SAM · Xi'an | IoU: 0.173 -> 0.747 | Helped |
| add mask prompt (ablation) | UV-SAM · Beijing | IoU: 0.635 -> 0.721 | Helped |
| add mask prompt (ablation) | UV-SAM · Xi'an | IoU: 0.744 -> 0.747 | Helped |
| add SAM image embedding (ablation) | UV-SAM · Beijing | IoU: 0.697 -> 0.721 | Helped |
| add SAM image embedding (ablation) | UV-SAM · Xi'an | IoU: 0.717 -> 0.747 | Helped |
| add SegFormer image embedding (ablation) | UV-SAM · Beijing | IoU: 0.694 -> 0.721 | Helped |
| add SegFormer image embedding (ablation) | UV-SAM · Xi'an | IoU: 0.733 -> 0.747 | Helped |
Surprising findings
- Removing the box prompt causes a catastrophic IoU drop on Xi'an (0.173), while the same ablation on Beijing shows only a small drop.
- RSPrompter, a SAM-based prompting method, performs worse than classical CNN segmentation baselines on urban village identification.
Limitations
- Reduced interpretability of results, as acknowledged by the authors.
- Pre-classification step needed for spatial and temporal analysis, with low precision on Xi'an (AUC 0.756).
Experimental setup
Datasets: Beijing satellite image dataset (2,491 images, 2016) Xi'an satellite image dataset (837 images, 2018)
Benchmarks: Beijing urban village dataset Xi'an urban village dataset
Metrics: IoU F1-score Recall Precision human mask quality rating (1-10)
Baselines: FCN DeepLabv3+ UVLens RSPrompter
Builds on: Segment Anything Model SegFormer RSPrompter
Closest papers by meaning
- UrbanVLP: Multi-Granularity Vision-Language Pretraining for Urban Socioeconomic Indicator Prediction (AAAI 2025 · similarity 0.84)
- CityLens: Evaluating Large Vision-Language Models for Urban Socioeconomic Sensing (ICLR 2026 · similarity 0.84)
- UrbanDataLayer: A Unified Data Pipeline for Urban Science (NeurIPS 2024 · similarity 0.84)
- DynamicVL: Benchmarking Multimodal Large Language Models for Dynamic City Understanding (NeurIPS 2025 · similarity 0.83)
- UrbanFeel：A Comprehensive Benchmark for Temporal and Perceptual Understanding of City Scenes through Human Perspective (ICLR 2026 · similarity 0.83)
- FUSU: A Multi-temporal-source Land Use Change Segmentation Dataset for Fine-grained Urban Semantic Understanding (NeurIPS 2024 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.