Research map / Vision-language and multimodal / Multimodal LLMs
GeoChat: Grounded Large Vision-Language Model for Remote Sensing
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Builds GeoChat, a LoRA-finetuned LLaVA-1.5-style remote sensing vision-language model with region input and grounded output, trained on a 318k RS multimodal instruction dataset.
Problem
General-domain vision-language models perform poorly on remote sensing imagery, hallucinating on RS-specific queries and lacking region-level reasoning, visual grounding, and RS instruction-following data.
Approach
Unifies image-level, region-level and grounded conversation in one RS VLM via task-specific tokens and textual box coordinates, and builds an RS instruction dataset by generating conversations from object-detection annotations and attributes using text-only Vicuna-v1.5.
Key results
- Zero-shot scene classification: 84.43% on UCMerced and 72.03% on AID, above LLaVA-1.5 (68.00 / 51.00) and Qwen-VL (62.90 / 52.60).
- RSVQA-HRBEN zero-shot average accuracy 72.30%, versus 68.40 for LLaVA-1.5 and 63.06 for Qwen-VL.
- Region captioning ROUGE-L 87.2 versus 31.2 for MiniGPT-4-v2 on the authors' benchmark.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use GeoChat instead of LLaVA-1.5 | Remote-sensing scene classification · UCMerced | accuracy: 68.00 -> 84.43 | Helped |
| Use GeoChat instead of LLaVA-1.5 | Remote-sensing scene classification · AID | accuracy: 51.00 -> 72.03 | Helped |
| Use GeoChat instead of RSGPT | Remote-sensing VQA · RSVQA-LRBEN, presence | accuracy: 91.17 -> 91.09 | Hurt |
| Use GeoChat instead of RSGPT | Remote-sensing VQA · RSVQA-LRBEN, comparison | accuracy: 91.70 -> 90.33 | Hurt |
| Use GeoChat instead of RSGPT | Remote-sensing VQA · RSVQA-LRBEN, rural/urban | accuracy: 94.00 -> 94.00 | No clear effect |
| Use GeoChat instead of RSGPT | Remote-sensing VQA · RSVQA-LRBEN | average accuracy: 92.29 -> 90.70 | Hurt |
| Use GeoChat instead of MiniGPTv2 | Remote-sensing grounding · Grounding benchmark, small objects | acc@0.5: 1.7 -> 2.9 | Helped |
| Use GeoChat instead of MiniGPTv2 | Remote-sensing grounding · Grounding benchmark, medium objects | acc@0.5: 9.9 -> 13.6 | Helped |
| Use GeoChat instead of MiniGPTv2 | Remote-sensing grounding · Grounding benchmark, large objects | acc@0.5: 21.9 -> 21.7 | Hurt |
| Use GeoChat instead of MiniGPTv2 | Remote-sensing grounding · Grounding benchmark, overall | acc@0.5: 7.6 -> 10.6 | Helped |
| Use GeoChat instead of LLaVA-1.5 | Remote-sensing VQA · RSVQA-HRBEN, presence | accuracy: 69.83 -> 58.45 | Hurt |
| Use GeoChat instead of LLaVA-1.5 | Remote-sensing VQA · RSVQA-HRBEN, comparison | accuracy: 67.29 -> 83.19 | Helped |
Surprising findings
- MiniGPT-4-v2 often ignores the classification instruction and returns classes not in the dataset, yielding near 5% accuracy under a checking protocol.
Limitations
- Performance on small objects and multi-object grounding is low (e.g., 2.9% small single-object acc@0.5).
- Grounding benchmark is self-constructed from SAMRS validation data, with pseudo labels from ViTAE-RVSA for missing classes.
- Scene classification and VQA comparisons are zero-shot against generic VLMs; GeoChat trails supervised RSGPT on RSVQA-LRBEN.
Experimental setup
Datasets: SAMRS (DOTA, DIOR, FAIR1M) NWPU-RESISC-45 RSVQA-LRBEN Floodnet LoveDA
Benchmarks: AID UCMerced RSVQA-LRBEN RSVQA-HRBEN GeoChat grounding benchmark (derived from SAMRS val)
Metrics: accuracy acc@0.5 acc@0.25 METEOR ROUGE-1 ROUGE-L
Baselines: LLaVA-1.5 Qwen-VL MiniGPT-4-v2 RSGPT RSVQA EasyToHard Bi-Modal SHRNet
Builds on: LLaVA-1.5 Vicuna-v1.5 LoRA SAMRS
Closest papers by meaning
- GeoPixel: Pixel Grounding Large Multimodal Model in Remote Sensing (ICML 2025 · similarity 0.92)
- GeoLLaVA-8K: Scaling Remote-Sensing Multimodal Large Language Models to 8K Resolution (NeurIPS 2025 · similarity 0.91)
- VRSBench: A Versatile Vision-Language Benchmark Dataset for Remote Sensing Image Understanding (NeurIPS 2024 · similarity 0.91)
- VLRS-Bench: A Vision-Language Reasoning Benchmark for Remote Sensing (NeurIPS 2026 · similarity 0.90)
- GeoViS: Geospatially Rewarded Visual Search for Remote Sensing Visual Grounding (CVPR 2026 · similarity 0.90)
- Remote Sensing Vision-Language Foundation Models without Annotations via Ground Remote Alignment (ICLR 2024 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.