Research map / Vision-language and multimodal / Cross-modal learning
Unifying Heterogeneous Multi-Modal Remote Sensing Detection Via Language-Pivoted Pretraining
Yuxuan Li, Yuming Chen, Yunheng Li, Ming-Ming Cheng, Xiang Li, jian Yang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| language-pivoted early alignment pretraining instead of late-alignment fine-tuning (vs SM3Det) | BabelRS · SOI-Det overall | mAP: 50.20 -> 51.57 | Improved |
| language-pivoted pretraining vs generic backbones with late alignment | BabelRS vs DINOv2 ViT-Large · SOI-Det overall | mAP: 45.02 -> 51.57 | Improved |
| use language-pivoted pretraining under AMP fine-tuning | BabelRS · SOI-Det (AMP) | stability / mAP: MAE, BEiTv2, DINOv2, ScaleMAE NaN; BabelRS 50.17 mAP | Improved |
| CLIP-style global alignment pretraining | CLIP / RemoteCLIP · SOI-Det overall | mAP: 36.12 (CLIP), 37.17 (RemoteCLIP) vs 51.57 | Hurt |
| fuse intermediate ViT layers via LVSA with shared projector vs final-layer only (ablation) | BabelRS · SOI-Det | mAP: 49.33 -> 51.57 | Improved |
| naive feature concatenation of intermediate layers (ablation) | BabelRS · SOI-Det | mAP: 49.33 -> 50.25 | No clear effect |
| element-wise summation of intermediate layers (ablation) | BabelRS · SOI-Det | mAP: 49.33 -> 50.31 | No clear effect |
| per-layer projectors with LVSA (ablation) | BabelRS · SOI-Det | mAP: 49.33 -> 49.88 | No clear effect |
| increase annealing duration tau from 0 to 6k steps (ablation) | BabelRS LVSA · SOI-Det (AMP) | mAP, H-mAP | Improved |
| increase annealing duration beyond 6k (ablation) | BabelRS LVSA · SOI-Det (AMP) | mAP, H-mAP | No clear effect |
| pretrain beyond about 20k steps (ablation) | BabelRS · SOI-Det | mAP, H-mAP | Hurt |
Earlier methods this paper compares with
Named baselines from earlier accepted papers in this paper's results tables: better than 2 of 2 on most shared results.
- SkySense — SkySense: A Multi-Modal Remote Sensing Foundation Model Towards Universal Interpretation for Earth Observation Imagery (CVPR 2024) · better on 1 of 1 shared result
- BEiT — BEiT: BERT Pre-Training of Image Transformers (ICLR 2022) · better on 1 of 1 shared result
Matched by method name in results tables read by AI. Authors' own tables tend to favour the newer method, and a comparison can be missed when a method is renamed.
Closest papers by meaning
- SM3Det: A Unified Model for Multi-Modal Remote Sensing Object Detection (AAAI 2026 · similarity 0.90)
- T-APT: Text-Guided Modality-Aware Prompt Tuning for Arbitrary Multimodal Remote Sensing Data Joint Classification (AAAI 2026 · similarity 0.89)
- From Visual Primitives to Semantic Masks: Fine-Grained Visual-Linguistic Alignment for Open-Vocabulary Remote Sensing Image Segmentation (ECCV 2026 · similarity 0.89)
- Remote Sensing Vision-Language Foundation Models without Annotations via Ground Remote Alignment (ICLR 2024 · similarity 0.87)
- VLRS-Bench: A Vision-Language Reasoning Benchmark for Remote Sensing (NeurIPS 2026 · similarity 0.86)
- SkySense: A Multi-Modal Remote Sensing Foundation Model Towards Universal Interpretation for Earth Observation Imagery (CVPR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
