Research map / Detection and segmentation / CNN features
Pseudo Visible Feature Fine-Grained Fusion for Thermal Object Detection
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes PFGF, a graph-Mamba fusion method that fuses thermal multi-level features with pseudo-visible latent features from a Pearl-GAN thermal-to-visible translator for thermal-only object detection.
Problem
Thermal object detection at test time uses only thermal images, but existing T2V-based methods fuse pseudo-visible information with simple attention, failing to exploit complementary visible information and fine-grained cross-level relations.
Approach
Builds a graph of subgraphs (one per feature level plus the pseudo-visible latent code), fuses pseudo-visible and thermal nodes with an Inter-Mamba block at the lowest level, and propagates knowledge up levels via leader nodes in a Cascade Knowledge Integration strategy, with GRU node updates and VMamba refinement.
Key results
- On FLIR, PFGF reaches 47.1 mAP and 84.8 mAP50, beating TIRDet (44.3 mAP, 81.4 mAP50) among thermal-only methods.
- On LLVIP, PFGF achieves 67.3 mAP and 96.9 mAP50, and on the Autonomous Vehicles dataset 45.9 mAP versus 43.1 for IDA.
- Ablation shows removing Pearl-GAN drops FLIR mAP from 47.1 to 45.2 and removing the whole GMF drops it to 45.6.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add Pearl-GAN visible-information input (ablation) | PFGF · FLIR | mAP: 45.2 -> 47.1 | Helped |
| add Pearl-GAN visible-information input (ablation) | PFGF · LLVIP | mAP: 65.0 -> 67.3 | Helped |
| add Pearl-GAN visible-information input (ablation) | PFGF · Autonomous Vehicles | mAP: 44.4 -> 45.9 | Helped |
| add VMamba-based Multi-level Feature Enhancement (ablation) | PFGF · FLIR | mAP: 46.0 -> 47.1 | Helped |
| add VMamba-based Multi-level Feature Enhancement (ablation) | PFGF · LLVIP | mAP: 66.1 -> 67.3 | Helped |
| add VMamba-based Multi-level Feature Enhancement (ablation) | PFGF · Autonomous Vehicles | mAP: 45.4 -> 45.9 | Helped |
| add Graph-Mamba Fusion module (ablation) | PFGF · FLIR | mAP: 45.6 -> 47.1 | Helped |
| add Graph-Mamba Fusion module (ablation) | PFGF · LLVIP | mAP: 65.6 -> 67.3 | Helped |
| add Graph-Mamba Fusion module (ablation) | PFGF · Autonomous Vehicles | mAP: 44.3 -> 45.9 | Helped |
| add Inter-Mamba cross-modality fusion (ablation) | PFGF · FLIR | mAP: 45.9 -> 47.1 | Helped |
| add Inter-Mamba cross-modality fusion (ablation) | PFGF · LLVIP | mAP: 66.2 -> 67.3 | Helped |
| add Inter-Mamba cross-modality fusion (ablation) | PFGF · Autonomous Vehicles | mAP: 45.1 -> 45.9 | Helped |
Surprising findings
- Fusion-Mamba-style multispectral methods were not better than the proposed thermal-only graph fusion on mAP75 and mAP, and the Mamba-based GMF outperforms its transformer counterpart GMTF with fewer FLOPs.
Limitations
- Relies on the quality of the Pearl-GAN translated visible images, which the authors note can be unreliable and noisy.
- Fusion with the generated visible information requires a two-stage training pipeline with a separately trained T2V model.
Experimental setup
Datasets: FLIR-aligned LLVIP Autonomous Vehicles
Benchmarks: FLIR-aligned LLVIP Autonomous Vehicles
Metrics: mAP mAP50 mAP75
Baselines: Faster RCNN SSD RetinaNet YOLOv3 YOLOX-L IAH DIP TIRDet IDA CFR GAFF YOLOFusion
Builds on: YOLOX Pearl-GAN VMamba TIRDet GIM
Closest papers by meaning
- DiMSOD: A Diffusion-Based Framework for Multi-Modal Salient Object Detection (AAAI 2025 · similarity 0.85)
- DetectiumFire: A Comprehensive Multi-modal Dataset Bridging Vision and Language for Fire Understanding (NeurIPS 2025 · similarity 0.83)
- Cocoon: Robust Multi-Modal Perception with Uncertainty-Aware Sensor Fusion (ICLR 2025 · similarity 0.83)
- Alignment-Free RGB-T Salient Object Detection: A Large-Scale Dataset and Progressive Correlation Network (AAAI 2025 · similarity 0.83)
- SSLFusion: Scale and Space Aligned Latent Fusion Model for Multimodal 3D Object Detection (AAAI 2025 · similarity 0.83)
- SMR-Net: Semantic-Guided Mutually Reinforcing Network for Cross-Modal Image Fusion and Salient Object Detection (AAAI 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.