Research map / Vision-language and multimodal / Human and embodied modeling
Sim2Real-Fire: A Multi-modal Simulation Dataset for Forecast and Backtracking of Real-world Forest Fire
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces Sim2Real-Fire, a dataset of 1M simulated multi-modal wildfire scenarios (FARSITE, WFDS, WRF-SFIRE) plus 1K annotated real satellite wildfire scenarios, and S2R-FireTr, a Swin-based transformer trained on simulation for wildfire forecast and backtracking.
Problem
Training AI wildfire forecast and backtracking models requires large, temporally complete multi-modal data, which real satellite observations lack; models trained on simulated fires suffer a Sim2Real gap when applied to real wildfires.
Approach
Use cheap simulator-generated wildfire scenarios from multiple simulators with jittered parameters as training data, and a dual cross-attention transformer (spatial and temporal) fusing topography, vegetation, fuel, weather and satellite inputs with time-wise regression over possibly irregular timestamps to reduce the Sim2Real gap.
Key results
- S2R-FireTr reaches 72.9 AUPRC on real-world forecasting, versus 63.0 for the best simulator (WRF-SFIRE) and 66.2 for the best AI baseline (PredRNN-v2).
- S2R-FireTr achieves 63.9 AUPRC on real-world backtracking, above Rainformer's 54.6.
- Removing either the environment-guided area representation learning or the time-wise regression module lowers real-world forecast AUPRC (e.g., 60.2 without both vs 72.9 full).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use S2R-FireTr instead of Rainformer | wildfire forecast · Forecast, real-world data | AUPRC: 67.2 -> 72.9 | Helped |
| use S2R-FireTr instead of Rainformer | wildfire backtracking · Backtrack, real-world data | AUPRC: 54.6 -> 63.9 | Helped |
| add environment-guided area representation learning (ablation) | S2R-FireTr · Forecast, real-world data | AUPRC: 60.2 -> 70.1 | Helped |
| add time-wise fire area regression (ablation) | S2R-FireTr · Forecast, real-world data | AUPRC: 60.2 -> 63.9 | Helped |
| add environment-guided area representation learning (ablation) | S2R-FireTr · Backtrack, real-world data | AUPRC: 42.4 -> 59.0 | Helped |
| add time-wise fire area regression (ablation) | S2R-FireTr · Backtrack, real-world data | AUPRC: 42.4 -> 58.5 | Helped |
| remove topography, vegetation, fuel, weather, or satellite modality (ablation) | S2R-FireTr · Forecast, real-world data | AUPRC: 72.9 -> 66.9; 72.9 -> 67.7; 72.9 -> 69.3; 72.9 -> 67.0; 72.9 -> 67.4 | Hurt |
| vary temporal sequence length from 3 to 4 frames (ablation) | S2R-FireTr · Forecast, real-world data | AUPRC: 72.9 -> 70.6 | Hurt |
Surprising findings
- Simulators such as WRF-SFIRE and WFDS, which need no training, underperform trained AI models on real-world forecasting in the reported comparison, yet all AI models degrade substantially from simulation to real data.
Limitations
- Dataset covers only certain countries and periods due to limited acquisition budget, reducing generalization to unseen environments.
- Performance degrades for long input/output sequences; the model is best within about six frames.
- Real-world test set is small (1K scenarios) and annotated manually by 20 annotators.
Experimental setup
Datasets: Sim2Real-Fire simulation data (1M scenarios from FARSITE, WFDS, WRF-SFIRE) Sim2Real-Fire real-world data (1K scenarios from Landsat-8 and Sentinel-2)
Benchmarks: Sim2Real-Fire simulation test split Sim2Real-Fire real-world test set (1K scenarios)
Metrics: AUPRC F1 IOU
Baselines: ConvLSTM MAU PredRNN-v2 Rainformer Earthformer SwinLSTM EarthFarseer ML-BPM OLDM FARSITE WFDS WRF-SFIRE
Builds on: Swin Transformer FARSITE WFDS WRF-SFIRE
Closest papers by meaning
- FireScope: Wildfire Risk Raster Prediction With a Chain-of-Thought Oracle (CVPR 2026 · similarity 0.85)
- DetectiumFire: A Comprehensive Multi-modal Dataset Bridging Vision and Language for Fire Understanding (NeurIPS 2025 · similarity 0.84)
- SNN-PDE: Learning Dynamic PDEs from Data with Simplicial Neural Networks (AAAI 2024 · similarity 0.81)
- Multi-modal Learning for Geospatial Vegetation Forecasting (CVPR 2024 · similarity 0.80)
- Fire360: A Benchmark for Robust Perception and Episodic Memory in Degraded 360° Firefighting Video (NeurIPS 2025 · similarity 0.79)
- HDT: Hierarchical Discrete Transformer for Multivariate Time Series Forecasting (AAAI 2025 · similarity 0.79)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.