Research map / Vision-language and multimodal / Multimodal LLMs

SkyMoE: A Vision-Language Foundation Model for Enhancing Geospatial Interpretation with Mixture of Experts

AAAI 2026 · Vision-language and multimodal › Multimodal LLMs · approach: VLM capability analysis · 2 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes SkyMoE, a mixture-of-experts vision-language model with context-disentangled augmentation for multi-granularity remote sensing interpretation, and introduces the MGRS-Bench benchmark.

Problem

General and remote-sensing VLMs use monolithic architectures that struggle to balance fine-grained local object perception with global scene understanding across diverse RS tasks.

Approach

An MoE LLM whose experts are cloned from dense FFNs and trained with context-disentangled augmentations (count-varying cutout and attribute editing) to encourage local/global expert specialization, plus a new multi-granularity benchmark MGRS-Bench.

Key results

• SkyMoE reaches 68.6% mAP@0.5 on DIOR-RSVG visual grounding, versus 60.8% for the best listed baseline (SkySenseGPT).
• SkyMoE scores 93.13% on the RSVQA-based VQA ablation and 98.11% on RSVQA-LR rural classification, the top result in its VQA table.
• Ablation: VG rises from 45.4% (1 expert) to 68.6% (8 experts); removing both augmentations drops VG to 60.2% and OC on RSOD to 56.37%.

Surprising findings

• SkyMoE is not best on scene classification, trailing top models by about 2% on AID and WHU-RS19, which the authors attribute to a trade-off against global semantics.

Limitations

• Scene classification trails top baselines by roughly 2%.
• Ablation results show the expert count and augmentation gains are not fully disentangled from training-data changes.

Experimental setup

Datasets: DOTA DIOR FAIR1M DOTA-v2 LAION-CC-SBU 558K subset DIOR-RSVG RSOD NWPU-RESISC45

Benchmarks: UCM-Captions RSICD RSITMD NWPU-Captions Sydney-Captions MGRS-Bench RSVQA-LR RSVQA-HR DIOR-RSVG AVVG RRSIS-D RSVG

Metrics: BLEU-4 METEOR ROUGE-L accuracy mAP@0.5 mAP@0.7 per-token latency TFLOPs

Baselines: GPT-4V MiniGPT-v2 LLaVA-1.5 Qwen2.5-VL DeepSeek-VL GeoChat VHM SkySenseGPT LHRS-Bot RSUniVLM EarthDial Falcon

Builds on: LLaVA Vicuna CLIP LoRA MoE-LLaVA

Closest papers by meaning

• SkySense-O: Towards Open-World Remote Sensing Interpretation with Vision-Centric Visual-Language Modeling (CVPR 2025 · similarity 0.91)
• Towards Faithful Reasoning in Remote Sensing: A Perceptually-Grounded GeoSpatial Chain-of-Thought for Vision-Language Models (ICLR 2026 · similarity 0.90)
• GeoCoT: Towards Reliable Remote Sensing Reasoning with Manifold Perspective (CVPR 2026 · similarity 0.90)
• GeoViS: Geospatially Rewarded Visual Search for Remote Sensing Visual Grounding (CVPR 2026 · similarity 0.90)
• SkyScript: A Large and Semantically Diverse Vision-Language Dataset for Remote Sensing (AAAI 2024 · similarity 0.89)
• RemoteReasoner: Towards Unifying Geospatial Reasoning Workflow (AAAI 2026 · similarity 0.89)