Research map / Vision-language and multimodal / Multimodal LLMs
DisasterM3: A Remote Sensing Vision-Language Dataset for Disaster Damage Assessment and Response
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces DisasterM3, a multi-hazard, multi-sensor (optical and SAR) remote sensing vision-language benchmark and instruction dataset for disaster damage assessment and response, benchmarks 14 VLMs, and fine-tunes four of them.
Problem
Existing remote sensing VLMs and datasets do not cover complex disaster scenes with diverse hazards, geographies, and sensors (optical and SAR), so their reasoning for disaster assessment and response is weak.
Approach
A bi-temporal optical/SAR disaster dataset with 9 tasks spanning recognition, counting, referring segmentation, relational reasoning, and report generation, plus LoRA/SFT fine-tuning of open VLMs and segmentation models on its Instruct split to produce baselines.
Key results
- Fine-tuned Qwen2.5-VL-7B and InternVL3-8B improve over base models on DisasterM3 Bench optical-optical QA; InternVL3-8B gains +10.4 AVG accuracy (31.3 to 41.7).
- Fine-tuned PSALM-1.3B reaches 50.5 mIoU on optical-optical referring segmentation, up 40.8 over the base model.
- Best zero-shot model GPT-4.1 reaches 42.3% AVG accuracy, while SAR-setting performance drops substantially across all models.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Fine-tune Qwen2.5-VL-7B on DisasterM3 Instruct | Qwen2.5-VL-7B · DisasterM3 Bench, optical-optical | AVG accuracy (%): ↑9.2 | Helped |
| Fine-tune InternVL3-8B on DisasterM3 Instruct | InternVL3-8B · DisasterM3 Bench, optical-optical | AVG accuracy (%): ↑10.4 | Helped |
| Fine-tune InternVL3-8B on DisasterM3 Instruct | InternVL3-8B · DisasterM3 Bench, optical-optical | BDC accuracy (%): ↓-1.2 | Hurt |
| Fine-tune Qwen2.5-VL-7B on DisasterM3 Instruct | Qwen2.5-VL-7B · DisasterM3 Bench, optical-optical | Disaster Caption AVG: ↑2.15 | Helped |
| Fine-tune InternVL3-8B on DisasterM3 Instruct | InternVL3-8B · DisasterM3 Bench, optical-optical | Restoration Advice AVG: ↑0.56 | Helped |
| Fine-tune LISA-7B on DisasterM3 Instruct | LISA-7B · Optical-optical referring segmentation | mIoU (%): ↑17.3 | Helped |
| Fine-tune PSALM-1.3B on DisasterM3 Instruct | PSALM-1.3B · Optical-optical referring segmentation | mIoU (%): ↑40.8 | Helped |
| Fine-tune LISA-7B on DisasterM3 Instruct | LISA-7B · Optical-SAR referring segmentation | mIoU (%): ↑17.3 | Helped |
| Fine-tune PSALM-1.3B on DisasterM3 Instruct | PSALM-1.3B · Optical-SAR referring segmentation | mIoU (%): ↑23.7 | Helped |
| Fine-tune Qwen2.5-VL-7B on DisasterM3 Instruct | Qwen2.5-VL-7B · DisasterM3 Bench, optical-SAR | Disaster Caption AVG: 0.98 compared to 3.65 (3.7× improvement) | Helped |
| Increase InternVL3 model size | InternVL3 series · DisasterM3 Bench, optical-optical | AVG accuracy (%) | Helped |
| Increase Qwen2.5-VL model size | Qwen2.5-VL series · DisasterM3 Bench, optical-SAR | AVG accuracy (%) | No clear effect |
Surprising findings
- Fine-tuned InternVL3-8B degrades on building damage counting (BDC), attributed to overfitting.
- Larger Qwen2.5-VL models do not yield stable gains in the optical-SAR setting.
Limitations
- Fixed 0.8 m resolution and single-polarization SAR limit generalization to other resolutions and sensors.
- Counting performance overfits after fine-tuning, with degradation at higher building densities.
- Open-ended scoring relies on GPT-4.1 as judge.
Experimental setup
Datasets: xBD BRIGHT AID
Benchmarks: DisasterM3 Bench
Metrics: accuracy GPT-4.1 5-point score (DAP, DDR, FC, RN, APP, SC) mIoU cIoU
Baselines: LLaVA-OneVision InternVL3 Kimi-VL Qwen2.5-VL GPT-4o GPT-4.1 GeoChat TeoChat EarthDial LISA PSALM HyperSeg
Builds on: LISA PSALM xBD
Closest papers by meaning
- RSCC: A Large-Scale Remote Sensing Change Caption Dataset for Disaster Events (NeurIPS 2025 · similarity 0.90)
- MONITRS: Multimodal Observations of Natural Incidents Through Remote Sensing (NeurIPS 2025 · similarity 0.88)
- Generalizable Disaster Damage Assessment via Change Detection with Vision Foundation Model (AAAI 2025 · similarity 0.87)
- Can LLM Agents Respond to Disasters? Benchmarking Heterogeneous Geospatial Reasoning in Emergency Operations (NeurIPS 2026 · similarity 0.87)
- CHOICE: Benchmarking the Remote Sensing Capabilities of Large Vision-Language Models (NeurIPS 2025 · similarity 0.87)
- SkyScript: A Large and Semantically Diverse Vision-Language Dataset for Remote Sensing (AAAI 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.