Research map / Vision-language and multimodal / CLIP and zero-shot
ZoRI: Towards Discriminative Zero-Shot Remote Sensing Instance Segmentation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes zero-shot remote sensing instance segmentation (ZoRI), adapting a FC-CLIP-style open-vocabulary segmenter with refined text-channel classifier, decoupled visual adaptation, and a visual-prototype cache bank, evaluated on iSAID and NWPU-VHR-10 splits.
Problem
Instance segmentation of remote sensing images is restricted to closed-set categories, and CLIP-based zero-shot segmentation transfers poorly to aerial imagery due to domain gap, high inter-class similarity and intra-class variance.
Approach
Selects text-embedding channels that minimize cross-class similarity and maximize variance (discrimination-enhanced classifier), freezes semantically aligned visual channels while tuning the rest (knowledge-maintained adaptation), and adds a training-free cache bank of aerial visual prototypes to the CLIP zero-shot logits (prior-injected prediction).
Key results
- On iSAID under GZSRI, ZoRI reaches 15.53 HM-mAP vs 8.83 for FC-CLIP and 1.26 for ZSI.
- On NWPU-VHR-10 under GZSRI, ZoRI reaches 21.28 HM-mAP vs 12.65 for FC-CLIP.
- Under ZSRI on iSAID, ZoRI reports 8.6 mAP vs 3.0 for FC-CLIP.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add ZoRI instead of FC-CLIP | iSAID GZSRI · iSAID GZSRI | HM mAP: 8.83 -> 15.53 | Helped |
| Add ZoRI instead of FC-CLIP | NWPU-VHR-10 GZSRI · NWPU-VHR-10 GZSRI | HM mAP: 12.65 -> 21.28 | Helped |
| Add ZoRI instead of FC-CLIP | iSAID ZSRI · iSAID ZSRI | mAP: 3.0 -> 8.6 | Helped |
| Add ZoRI instead of FC-CLIP | NWPU-VHR-10 ZSRI · NWPU-VHR-10 ZSRI | mAP: 4.5 -> 9.6 | Helped |
| Add DEC to the baseline (ablation) | FC-CLIP on iSAID · iSAID GZSRI | HM mAP: 8.83 -> 14.05 | Helped |
| Add DEC to the baseline (ablation) | FC-CLIP on NWPU-VHR-10 · NWPU-VHR-10 GZSRI | HM mAP: 12.65 -> 15.93 | Helped |
| Add KMA to DEC (ablation) | ZoRI on iSAID · iSAID GZSRI | HM mAP: 14.05 -> 14.55 | Helped |
| Add KMA to DEC (ablation) | ZoRI on NWPU-VHR-10 · NWPU-VHR-10 GZSRI | HM mAP: 15.93 -> 16.70 | Helped |
| Add PIP to DEC + KMA (ablation) | ZoRI on iSAID · iSAID GZSRI | HM mAP: 14.55 -> 15.53 | Helped |
| Add PIP to DEC + KMA (ablation) | ZoRI on NWPU-VHR-10 · NWPU-VHR-10 GZSRI | HM mAP: 16.70 -> 21.28 | Helped |
| Add PIP to DEC + KMA (ablation) | ZoRI on iSAID · iSAID GZSRI | HM Recall@100: 51.60 -> 48.76 | Hurt |
| Vary selected text-classifier channels; 300 outperforms 200 and 400–768 (ablation) | DEC on iSAID · iSAID GZSRI | HM mAP: 11.02; 14.06; 13.69; 12.26; 11.47; 10.86; 8.86 | Mixed |
Surprising findings
- Cache bank prior-injected prediction slightly lowers recall even as mAP improves, which the authors attribute to predictions being pulled toward cache samples.
- Full finetuning of the CLIP vision encoder yields only marginal unseen mAP gain over the frozen baseline (5.71 vs 4.91), suggesting overfitting to seen classes.
Limitations
- Evaluated only on three remote sensing datasets with fixed seen/unseen splits, and NWPU-VHR-10 splits were self-created by random split.
- Unseen-class performance remains low in absolute terms (e.g., 9.30 unseen mAP on iSAID).
- Recall drops after cache bank integration on some settings.
Experimental setup
Datasets: iSAID NWPU-VHR-10 SIOR
Benchmarks: iSAID NWPU-VHR-10 SIOR
Metrics: mAP Recall@100 harmonic mean
Baselines: ZSI D2Zero FC-CLIP VPT full finetuning
Builds on: FC-CLIP Mask2Former CLIP ZSI VPT
Closest papers by meaning
- Towards Open-Vocabulary Remote Sensing Image Semantic Segmentation (AAAI 2025 · similarity 0.88)
- ROS-SAM: High-Quality Interactive Segmentation for Remote Sensing Moving Object (CVPR 2025 · similarity 0.87)
- InstructSAM: A Training-free Framework for Instruction-Oriented Remote Sensing Object Recognition (NeurIPS 2025 · similarity 0.86)
- SkyScript: A Large and Semantically Diverse Vision-Language Dataset for Remote Sensing (AAAI 2024 · similarity 0.86)
- SkySense-O: Towards Open-World Remote Sensing Interpretation with Vision-Centric Visual-Language Modeling (CVPR 2025 · similarity 0.86)
- DiffCLIP: Few-shot Language-driven Multimodal Classifier (AAAI 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.