Research map / Vision-language and multimodal / Multimodal LLMs
DetectiumFire: A Comprehensive Multi-modal Dataset Bridging Vision and Language for Fire Understanding
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces DetectiumFire, a multi-modal fire dataset of real images/videos with bounding boxes and captions, plus SFT/DPO-tuned diffusion synthetic data and a fine-tuned LLaMA-3.2-11B-Vision for fire reasoning.
Problem
Fire understanding lacks large, public, high-quality multi-modal datasets with fire-domain annotations; existing datasets (e.g., D-Fire) are small, redundant, and limited to detection.
Approach
A real-world fire image/video dataset with bounding boxes, 75-token fire-specific captions (burning object, environment, severity), controlled and uncontrolled fire coverage, hard negatives, plus synthetic images from SFT and Diffusion-DPO fine-tuned diffusion models and a human preference set.
Key results
- YOLOv11m trained on DetectiumFire reaches 40.32% mAP on D-Fire test, close to 40.28% when trained on D-Fire itself, while D-Fire-trained models score 24.88% mAP on DetectiumFire.
- Duplication ratio via CNN similarity drops to 0.23 for DetectiumFire versus 0.55 for D-Fire.
- Fine-tuned LLaMA-3.2-11B-Vision raises fire severity accuracy from 56.06% to 83.84%, burning object from 62.73% to 87.37%, and environment from 71.82% to 89.39%.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Train YOLOv11m on DetectiumFire rather than D-Fire | YOLOv11m · DetectiumFire test | mAP: 24.88 ± 0.77 -> 43.74 ± 0.64 | Helped |
| Train YOLOv11m on DetectiumFire rather than D-Fire | YOLOv11m · D-Fire test | mAP: 40.28 ± 1.76 -> 40.32 ± 1.53 | No clear effect |
| Use DetectiumFire rather than D-Fire as the dataset | Fire image dataset · Images measured with PHash | duplication ratio: 0.15 -> 0.03 | Helped |
| Use DetectiumFire rather than D-Fire as the dataset | Fire image dataset · Images measured with CNN similarity | duplication ratio: 0.55 -> 0.23 | Helped |
| Fine-tune LLaMA-3.2-11B-Vision-Instruct on DetectiumFire image-caption pairs | LLaMA-3.2-11B-Vision-Instruct · Fire reasoning: burning object | accuracy: 62.73% -> 87.37% | Helped |
| Fine-tune LLaMA-3.2-11B-Vision-Instruct on DetectiumFire image-caption pairs | LLaMA-3.2-11B-Vision-Instruct · Fire reasoning: environment | accuracy: 71.82% -> 89.39% | Helped |
| Fine-tune LLaMA-3.2-11B-Vision-Instruct on DetectiumFire image-caption pairs | LLaMA-3.2-11B-Vision-Instruct · Fire reasoning: fire severity | accuracy: 56.06% -> 83.84% | Helped |
| Train on synthetic images generated using SFT instead of RLHF | YOLOv11m · DetectiumFire test | mAP: 32.12±0.29 -> 33.50±0.35 | Helped |
| Train on SFT-generated synthetic images rather than FLAME_SD | YOLOv11m · DetectiumFire test | mAP: 2.10±3.90 -> 33.50±0.35 | Helped |
| Add all synthetic training data to real-world training data | YOLOv11m · DetectiumFire test | mAP: 43.74±0.64 -> 44.52±0.50 | Helped |
| Add all synthetic training data to real-world training data | Faster R-CNN · DetectiumFire test | mAP: 41.13±1.01 -> 42.56±2.44 | Helped |
| Train on synthetic SFT data rather than real-world data | YOLOv11m · DetectiumFire test | mAP: 43.74±0.64 -> 33.50±0.35 | Hurt |
Surprising findings
- Models trained on FLAME_SD synthetic images perform drastically worse (e.g., 2.10% mAP for YOLOv11m), while SFT/RLHF synthetic data reaches about 33% mAP.
- RLHF-generated synthetic data yields slightly lower detection performance than SFT-generated data, which the authors attribute to reduced diversity.
Limitations
- Much of the real-world data is web-sourced and restricted to academic use, limiting downstream applicability.
- Captions are limited to 75 tokens and exclude people-presence and temporal information.
- Coverage of geographies and culturally specific fire scenarios is not exhaustive.
Experimental setup
Datasets: DetectiumFire D-Fire FIRE Dataset Forest Fire dataset FireNET FLAME_SD
Benchmarks: DetectiumFire test set D-Fire
Metrics: mAP mAP@50 mAR@100 duplication ratio Elo score accuracy
Baselines: D-Fire Faster R-CNN YOLO-World-M FLAME_SD LLaMA-3.2-11B-Vision-Instruct
Builds on: D-Fire Diffusion-DPO FLAME YOLOv11 LoRA
Closest papers by meaning
- Fire360: A Benchmark for Robust Perception and Episodic Memory in Degraded 360° Firefighting Video (NeurIPS 2025 · similarity 0.88)
- FireScope: Wildfire Risk Raster Prediction With a Chain-of-Thought Oracle (CVPR 2026 · similarity 0.85)
- Sim2Real-Fire: A Multi-modal Simulation Dataset for Forecast and Backtracking of Real-world Forest Fire (NeurIPS 2024 · similarity 0.84)
- FIRE: A Dataset for Feedback Integration and Refinement Evaluation of Multimodal Models (NeurIPS 2024 · similarity 0.84)
- Pseudo Visible Feature Fine-Grained Fusion for Thermal Object Detection (CVPR 2025 · similarity 0.83)
- Roboflow100-VL: A Multi-Domain Object Detection Benchmark for Vision-Language Models (NeurIPS 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.