Research map / Vision-language and multimodal / Multimodal LLMs
VLRS-Bench: A Vision-Language Reasoning Benchmark for Remote Sensing
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces VLRS-Bench, a 2,000-item benchmark for complex remote sensing reasoning across Cognition, Decision, and Prediction, built with GPT-5-chat generation, multi-model verification, and expert review, and evaluates 20+ general and RS MLLMs.
Problem
Existing remote sensing benchmarks focus on perception tasks and lack cognitively demanding reasoning, temporal depth, and use of RS-specific priors such as DSM, NIR, and expert masks.
Approach
A cognition-driven three-level taxonomy (Cognition, Decision, Prediction; 6 L-2 abilities; 14 L-3 tasks) with long, RS-prior-enriched questions and up to 8 temporal images, generated automatically by GPT-5-chat and filtered by a three-stage verification pipeline including nine RS experts.
Key results
- GPT-5.4 achieves the highest general-model average (0.439), with Gemini-3.1-Pro-Preview close behind at 0.436 on VLRS-Bench.
- Single-choice and true/false questions average 53.1% and 47.6%, versus 15.8% for multi-choice and 9.0% for fill-in-the-blank.
- ScoreRS w/ RL (0.355) and w/ SFT (0.347) outperform the larger general Qwen2.5-VL-72B (0.264) in the RS-specialized group average comparison, and Qwen3-VL-32B scores 0.395.
Surprising findings
- Domain-specialized ScoreRS models outperform much larger general models such as Qwen2.5-VL-72B, suggesting scale alone does not drive geospatial reasoning.
- Models often score higher on Post-event evaluation than on Pre-event planning, showing asymmetry between generating and judging plans.
Limitations
- Benchmark construction relies heavily on GPT-5-chat generation, which may introduce generator-specific bias despite cross-model verification.
- Evaluation is zero-shot with a single standardized prompt, so prompt sensitivity is not analyzed.
Experimental setup
Datasets: LoveDA Potsdam Vaihingen GID15 DIOR DOTA FAIR1M Xview2 SECOND miniUCD SpaceNet7
Benchmarks: VLRS-Bench
Metrics: accuracy partial-credit score embedding similarity match
Baselines: GPT-5.4 Gemini-3.1-Pro-Preview Qwen3-VL-32B ScoreRS GeoChat VHM
Builds on: RSVQA CHOICE SAMRS
Closest papers by meaning
- VRSBench: A Versatile Vision-Language Benchmark Dataset for Remote Sensing Image Understanding (NeurIPS 2024 · similarity 0.93)
- Towards Faithful Reasoning in Remote Sensing: A Perceptually-Grounded GeoSpatial Chain-of-Thought for Vision-Language Models (ICLR 2026 · similarity 0.92)
- CHOICE: Benchmarking the Remote Sensing Capabilities of Large Vision-Language Models (NeurIPS 2025 · similarity 0.92)
- GeoCoT: Towards Reliable Remote Sensing Reasoning with Manifold Perspective (CVPR 2026 · similarity 0.91)
- Quality-Driven Curation of Remote Sensing Vision-Language Data via Learned Scoring Models (NeurIPS 2025 · similarity 0.91)
- GeoChat: Grounded Large Vision-Language Model for Remote Sensing (CVPR 2024 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.