Research map / Vision-language and multimodal / Human and embodied modeling
MONITRS: Multimodal Observations of Natural Incidents Through Remote Sensing
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Builds MONITRS, a dataset pairing ~10,000 FEMA disaster events with temporal Sentinel-2 imagery, news-derived captions, geolocations and VQA pairs, and fine-tunes TEOChat on it to beat baselines on disaster QA.
Problem
Automated satellite-image analysis of natural disasters is limited by narrow disaster-type coverage, reliance on expert interpretation, and a lack of temporally granular datasets with natural language annotations for tracking disaster progression.
Approach
Pair FEMA disaster declarations with news articles to geolocate event extents and generate date-by-date natural language descriptions, then pair these with Sentinel-2 image sequences and build multiple-choice and open-ended VQA for fine-tuning multimodal LLMs.
Key results
- Fine-tuned TEOChat reaches 88.69% event classification and 70.72% temporal grounding accuracy, versus roughly 49-50% and 11-18% for baselines.
- On generated multiple-choice VQA, the fine-tuned model reaches 52.18% accuracy versus 28-37% for baselines.
- Location grounding remains hard: the fine-tuned model reaches only 23.25% versus 13-17% for baselines.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Fine-tune TEOChat on MONITRS-QA | TEOChat 7B · MONITRS-QA event classification | accuracy: 50.07% -> 88.69% | Helped |
| Fine-tune TEOChat on MONITRS-QA | TEOChat 7B · MONITRS-QA temporal grounding | accuracy: 18.02% -> 70.72% | Helped |
| Fine-tune TEOChat on MONITRS-QA | TEOChat 7B · MONITRS-QA location grounding | accuracy: 17.11% -> 23.25% | Helped |
| Fine-tune TEOChat on MONITRS-QA | TEOChat 7B · Generated VQA multiple-choice | accuracy: 36.99% -> 52.18% | Helped |
| Fine-tune TEOChat on MONITRS-QA | TEOChat 7B · Generated VQA open-ended | BLEU-1: 0.3439 -> 0.4046 | Helped |
| Fine-tune TEOChat on MONITRS-QA | TEOChat 7B · Generated VQA open-ended | METEOR: 0.4736 -> 0.4912 | Helped |
| Fine-tune TEOChat on MONITRS-QA | TEOChat 7B · Generated VQA open-ended | ROUGE-L: 0.3951 -> 0.4275 | Helped |
| Fine-tune TEOChat on MONITRS-QA | TEOChat 7B · Generated VQA open-ended LLM evaluation | Overall: 3.08 -> 3.08 | No clear effect |
Surprising findings
- Gemini 2.0-flash scores below the open-source video and EO models on generated multiple-choice VQA (28.13% vs. about 37%).
Limitations
- Relies on FEMA records covering only U.S. disasters, limiting generalization to global events.
- Sentinel-2 at 10m resolution with ~5-day revisit may miss critical stages of rapidly evolving disasters.
- Captions generated by an LLM from news articles may not match what is visible in imagery; only RGB imagery is included.
Experimental setup
Datasets: FEMA Disaster Declarations Areas Google Search news articles Sentinel-2 RGB imagery MONITRS-QA
Benchmarks: MONITRS-QA
Metrics: accuracy BLEU-1 BLEU-2 BLEU-3 BLEU-4 METEOR ROUGE-L LLM-judge score
Baselines: Video-LLaVA 7B TEOChat 7B Gemini 2.0-flash
Builds on: TEOChat Video-LLaVA McNemar's test
Closest papers by meaning
- DisasterM3: A Remote Sensing Vision-Language Dataset for Disaster Damage Assessment and Response (NeurIPS 2025 · similarity 0.88)
- RSCC: A Large-Scale Remote Sensing Change Caption Dataset for Disaster Events (NeurIPS 2025 · similarity 0.87)
- Flood Insights: Integrating Remote and Social Sensing Data for Flood Exposure, Damage, and Urgent Needs Mapping (AAAI 2024 · similarity 0.86)
- Can LLM Agents Respond to Disasters? Benchmarking Heterogeneous Geospatial Reasoning in Emergency Operations (NeurIPS 2026 · similarity 0.85)
- DynamicVL: Benchmarking Multimodal Large Language Models for Dynamic City Understanding (NeurIPS 2025 · similarity 0.84)
- GeoLLaVA-8K: Scaling Remote-Sensing Multimodal Large Language Models to 8K Resolution (NeurIPS 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.