Research map / Vision-language and multimodal / Human and embodied modeling
VRSBench: A Versatile Vision-Language Benchmark Dataset for Remote Sensing Image Understanding
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces VRSBench, a human-verified remote sensing vision-language benchmark with detailed captions, object referring, and VQA built semi-automatically with GPT-4V from DOTA-v2 and DIOR, and benchmarks LVLMs on captioning, grounding and VQA.
Problem
Existing remote sensing vision-language datasets are single-task, lack detailed object information, or lack human quality control, and do not support unambiguous visual grounding or diverse open-ended VQA.
Approach
A semi-automatic pipeline that feeds object attributes extracted from detection labels (DOTA-v2, DIOR) plus image metadata into GPT-4V to generate captions, unique-object referring sentences, and QA pairs, followed by full human verification; the resulting unified benchmark covers three tasks on one set of images.
Key results
- LLaVA-1.5 fine-tuned on VRSBench achieves the best captioning BLEU-1 of 48.1 and CIDEr of 33.9.
- GeoChat fine-tuned on VRSBench reaches 49.8% Acc@0.5 for visual grounding, with GeoChat without fine-tuning at 12.9%.
- Mini-Gemini reaches the best VQA overall accuracy of 77.8%, while GeoChat without fine-tuning gets 40.8%.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Fine-tune GeoChat on VRSBench | GeoChat · VRSBench-Cap test | CIDEr: 0.4 -> 28.2 | Helped |
| Fine-tune GeoChat on VRSBench | GeoChat · VRSBench-Ref test, all objects | Acc@0.5: 12.9 -> 49.8 | Helped |
| Fine-tune GeoChat on VRSBench | GeoChat · VRSBench-VQA test | All accuracy: 40.8 -> 76.0 | Helped |
| Use oriented instead of horizontal bounding boxes (ablation) | GeoChat · VRSBench-Ref test, all objects | Acc@0.5: 49.8% -> 24.3% | Hurt |
Surprising findings
- GPT-4V performs markedly worse than fine-tuned models on visual grounding (5.1% Acc@0.5), attributed to missing object information in its prompts.
- MiniGPT-v2 is weaker overall on grounding but performs better on non-unique objects.
Limitations
- Dataset limited to RGB images; no infrared, multispectral, SAR or temporal data.
- Geographic diversity limited to regions covered by DOTA-v2 and DIOR.
- Even the best model fails to achieve satisfactory grounding accuracy on the benchmark.
Experimental setup
Datasets: DOTA-v2 DIOR VRSBench
Benchmarks: VRSBench-Cap VRSBench-Ref VRSBench-VQA
Metrics: BLEU-1 BLEU-2 BLEU-3 BLEU-4 METEOR ROUGE_L CIDEr CHAIR Acc@0.5 Acc@0.7
Baselines: GeoChat LLaVA-1.5 MiniGPT-v2 Mini-Gemini GPT-4V
Builds on: LLaVA-1.5 GeoChat LoRA
Closest papers by meaning
- VLRS-Bench: A Vision-Language Reasoning Benchmark for Remote Sensing (NeurIPS 2026 · similarity 0.93)
- Quality-Driven Curation of Remote Sensing Vision-Language Data via Learned Scoring Models (NeurIPS 2025 · similarity 0.91)
- SkySense-O: Towards Open-World Remote Sensing Interpretation with Vision-Centric Visual-Language Modeling (CVPR 2025 · similarity 0.91)
- VHM: Versatile and Honest Vision Language Model for Remote Sensing Image Analysis (AAAI 2025 · similarity 0.91)
- SkyScript: A Large and Semantically Diverse Vision-Language Dataset for Remote Sensing (AAAI 2024 · similarity 0.91)
- GeoChat: Grounded Large Vision-Language Model for Remote Sensing (CVPR 2024 · similarity 0.91)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.