Research map / Vision-language and multimodal / Cross-modal learning
SkySense-VITA: Towards Universal In-context Segmentation of Multi-modal Remote Sensing Imagery
Kang Wu, Lei Yu, Junwei Luo, Bo Dang, Junjian Zhang, Xiangyuan Cai, Hongwei Hu, Jingdong Chen, Yansheng Li
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add VITA-Former for prompt fusion (vs naive concat+conv fusion) (ablation) | SkySense-VITA · FloodNet | mIoU (fused): 47.96 -> 62.19 | Improved |
| add VITA-Former (ablation) | SkySense-VITA · ETCI-Flood | mIoU (fused): 20.26 -> 29.58 | Improved |
| add VITA-Former (ablation) | SkySense-VITA · ETCI-Flood | mIoU (visual-only): 27.63 -> 27.40 | No clear effect |
| add decoupled decoding (train visual, textual and fused branches jointly) (ablation) | SkySense-VITA · FloodNet | mIoU (textual-only): 34.57 -> 57.52 | Improved |
| add decoupled decoding (ablation) | SkySense-VITA · FloodNet | mIoU (fused): 59.31 -> 62.19 | Improved |
| add modality prototypes via cross-attention (ablation) | VITA-Former · ETCI-Flood | mIoU: 25.81 -> 29.58 | Improved |
| add modality prototypes by element-wise addition instead of cross-attention (ablation) | VITA-Former · ETCI-Flood | mIoU: 25.81 -> 14.39 | Hurt |
| add image-level optical-SAR alignment pretraining (ablation) | SAR encoder · ETCI-Flood | mIoU: 26.82 -> 29.58 | Improved |
| add image-level optical-SAR alignment pretraining (ablation) | SAR encoder · C2S-SAR | mIoU: 13.68 -> 16.82 | Improved |
| add Semantic Granularity Annealing (coarse-to-fine curriculum) (ablation) | pixel-level pretraining · iSAID | mIoU: 57.71 -> 59.89 | Improved |
| add Semantic Granularity Annealing (ablation) | pixel-level pretraining · FloodNet | mIoU: 61.03 -> 62.19 | Improved |
| use more precise visual prompt (point to box to mask) (ablation) | SkySense-VITA visual-only · iSAID | mIoU: 23.97 -> 56.07 | Improved |
Closest papers by meaning
- SkySense: A Multi-Modal Remote Sensing Foundation Model Towards Universal Interpretation for Earth Observation Imagery (CVPR 2024 · similarity 0.92)
- MM-OVSeg: Multimodal Optical-SAR Fusion for Open-Vocabulary Segmentation in Remote Sensing (CVPR 2026 · similarity 0.89)
- SkySense-O: Towards Open-World Remote Sensing Interpretation with Vision-Centric Visual-Language Modeling (CVPR 2025 · similarity 0.89)
- RobSense: A Robust Multi-modal Foundation Model for Remote Sensing with Static, Temporal, and Incomplete Data Adaptability (CVPR 2025 · similarity 0.89)
- Towards Open-Vocabulary Remote Sensing Image Semantic Segmentation (AAAI 2025 · similarity 0.87)
- SkyScript: A Large and Semantically Diverse Vision-Language Dataset for Remote Sensing (AAAI 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
