Research map / Vision-language and multimodal / Multimodal LLMs

SenseBench: A Benchmark for Remote Sensing Low-Level Visual Perception and Description in Large Vision-Language Models

NeurIPS 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Multimodal LLM benchmarks · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces SenseBench, a 10,815-instance zero-shot benchmark for remote sensing low-level perception (MCQ) and diagnostic description (LLM-judged), evaluating 29 VLMs and revealing domain-prior skew, multi-distortion collapse, fluency illusion, and perception-description inversion.

Problem

Whether current vision-language models can perceive and describe physics-driven low-level degradations in remote sensing imagery, which existing RS benchmarks and natural-image IQA benchmarks do not cover.

Approach

A physics-based hierarchical taxonomy (6 L-1, 22 L-2 distortions) with parameter-controlled synthesis yields traceable labels for objective MCQ perception (whether/what/how) and, via GPT-5.2 generation plus human verification, reference diagnostic reports scored on completeness, correctness, and faithfulness by Prometheus, covering both single-image and paired-image protocols.

Key results

• Qwen3VL-32B reaches 70.06% on general vs 56.40% on RS distortions in single-image perception, with no model in the balanced high-accuracy quadrant.
• Gemini-3.1-pro-preview drops from 79.68% (single) to 41.17% (multi-distortion) and EarthDial from 48.89% to 20.50%, showing multi-distortion collapse.
• Descriptions score high on Faithfulness but low on Correctness (e.g., Ovis2.5-9B Faithfulness 1.93 vs Correctness 1.44 single-image), indicating a fluency illusion.

Surprising findings

• Models that perform strongly on single-image perception (e.g., InternVL3.5-38B) can drop substantially on paired-image description, a perception-description inversion.
• Several RS-specialized VLMs collapse on paired-image description, with near-100% zero-score rates on completeness.

Limitations

• Descriptions are generated by GPT-5.2 and human-verified, so the reference reports may carry generator bias.
• Paired-image evaluation excludes models without multi-image interfaces, leaving some results as missing.
• Evaluation is zero-shot with a single deterministic decoding setting and one sample per instance.

Experimental setup

Datasets: Google Earth imagery (GEE tiles) RICE Dataset Sentinel-2 Natural Earth OpenStreetMap

Benchmarks: SenseBench

Metrics: exact-match accuracy completeness correctness faithfulness sum score

Baselines: Q-Instruct-LLaVA-7B Q-Instruct-LLaVA-13B DepictQA GeoChat TEOChat EarthDial LHRS-Bot LHRS-Bot-nova

Builds on: Q-Bench DepictQA

Closest papers by meaning

• VRSBench: A Versatile Vision-Language Benchmark Dataset for Remote Sensing Image Understanding (NeurIPS 2024 · similarity 0.88)
• ZoomEarth: Active Perception for Ultra-High-Resolution Geospatial Vision-Language Tasks (CVPR 2026 · similarity 0.88)
• Quality-Driven Curation of Remote Sensing Vision-Language Data via Learned Scoring Models (NeurIPS 2025 · similarity 0.88)
• Q-Bench: A Benchmark for General-Purpose Foundation Models on Low-level Vision (ICLR 2024 · similarity 0.87)
• Towards Faithful Reasoning in Remote Sensing: A Perceptually-Grounded GeoSpatial Chain-of-Thought for Vision-Language Models (ICLR 2026 · similarity 0.87)
• CHOICE: Benchmarking the Remote Sensing Capabilities of Large Vision-Language Models (NeurIPS 2025 · similarity 0.87)