Research map / Vision-language and multimodal / Visual question answering

RSRCC: A Remote Sensing Regional Change Comprehension Benchmark Constructed via Retrieval-Augmented Best-of-𝑁 Ranking

NeurIPS 2026 · Vision-language and multimodal › Visual question answering · approach: Attention and prior-robust VQA

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Builds RSRCC, a 126k-question localized remote-sensing change QA benchmark from LEVIR-CD via segmentation-based candidate extraction, SigLIP screening, and retrieval-augmented Best-of-N LLM validation.

Problem

Existing remote sensing change captioning datasets describe global image-pair differences, leaving localized, change-specific semantic reasoning unexplored; reliable fine-grained semantic supervision is costly to obtain.

Approach

A hierarchical semi-supervised curation pipeline: connected-component candidate regions from segmentation masks, image-text encoder screening, and retrieval-augmented Best-of-N LLM ranking to resolve ambiguous cases, followed by LLM-generated MCQ, yes/no, and open-ended questions.

Key results

• Human agreement of the full pipeline reaches 98%, versus 11% for encoder-only and 61% for segmentation-only curation.
• Gemini-2.5-Pro achieves 68.32% total accuracy on yes/no questions and 48.76% on multiple-choice; Gemma-3-4B reaches 46.81% and 37.98%.
• Gemini-2.5-Flash open-ended CIDEr total is 57.18 and SPICE is 21.05.

Surprising findings

• Encoder-only curation achieves only 11% human agreement, far below LLM-based zero-shot curation baselines.
• Cosine similarity matched human judgments at 100% agreement while L2, L1, and Wasserstein scored lower (94%, 93%, 84%).

Limitations

• Pipeline is tied to a predefined set of semantic classes and may underperform on novel or compositional changes.
• Built only from LEVIR-CD (512x512 building-change imagery), limiting diversity.
• Theoretical analyses rely on strong simplifying assumptions (e.g., i.i.d. retained samples, well-separated groups) and are not empirically verified.

Experimental setup

Datasets: LEVIR-CD

Benchmarks: RSRCC

Metrics: accuracy BLEU BERTScore CIDEr SPICE human agreement

Baselines: Gemini-2.5-Pro Gemma-3-27B Gemini-2.5-Flash Gemma-3-4B Encoder Only Segmentation Only

Builds on: LEVIR-CD SigLIP LEVIR-CC RSCC

Closest papers by meaning

• VLRS-Bench: A Vision-Language Reasoning Benchmark for Remote Sensing (NeurIPS 2026 · similarity 0.87)
• VRSBench: A Versatile Vision-Language Benchmark Dataset for Remote Sensing Image Understanding (NeurIPS 2024 · similarity 0.87)
• UniChange: Unifying Change Detection with Multimodal Large Language Model (CVPR 2026 · similarity 0.87)
• RSCC: A Large-Scale Remote Sensing Change Caption Dataset for Disaster Events (NeurIPS 2025 · similarity 0.87)
• The Change You Want To Detect: Semantic Change Detection In Earth Observation With Hybrid Data Generationf (CVPR 2025 · similarity 0.87)
• Quality-Driven Curation of Remote Sensing Vision-Language Data via Learned Scoring Models (NeurIPS 2025 · similarity 0.87)