Research map / Image generation / Autoregressive and latent generation
Revisiting Generative Infrared and Visible Image Fusion Based on Human Cognitive Laws
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes HCLFuse, a generative infrared-visible fusion method combining an optimal-transport-aligned multi-scale mask-regulated variational bottleneck encoder with a diffusion model guided by time-varying physical constraints (heat conduction, structure preservation, cross-modal consistency).
Problem
Infrared and visible image fusion struggles with balancing modality information, limited generative capability of existing generative fusion methods, lack of interpretability in modality selection, and strong dependence on data distribution and quality.
Approach
Combines an information-bottleneck variational encoder fed with optimal-transport-aligned infrared images with a diffusion sampler whose reverse steps are corrected by three physics-inspired constraints whose strength decays over timesteps (time-varying physical guidance).
Key results
- On MSRS, HCLFuse reports the highest AG (6.4355), SF (17.899), DF (7.6427) and EN (6.8704) among compared methods, with AG 69.87% above the second-best.
- On MSRS semantic segmentation (Mask2Former), HCLFuse achieves the highest mIoU of 89.72 versus 89.60 for Text-DiFuse.
- Ablation on MSRS shows removing TPG drops SD from 49.55 to 36.90 and AG from 6.436 to 5.521.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use HCLFuse instead of SwinFusion | infrared-visible image fusion · MSRS | AG: 3.7885 -> 6.4355 | Helped |
| Use HCLFuse instead of GIFNet | infrared-visible image fusion · TNO | AG: 4.9954 -> 7.2112 | Helped |
| Use HCLFuse instead of GIFNet | infrared-visible image fusion · FMB | AG: 5.0481 -> 6.9814 | Helped |
| Use HCLFuse instead of Text-DiFuse | Mask2Former semantic segmentation of fused MSRS images · MSRS | mIoU: 89.60 -> 89.72 | Helped |
| Add time-varying physical guidance (ablation) | HCLFuse · MSRS | AG: 5.521 -> 6.436 | Helped |
| Use the variational bottleneck encoder instead of a standard multi-scale encoder (ablation) | HCLFuse · MSRS | AG: 6.038 -> 6.436 | Helped |
| Add optimal transport alignment (ablation) | HCLFuse · MSRS | AG: 3.578 -> 6.436 | Helped |
| Use DDIM sampling (ablation) | HCLFuse · MSRS | AG: 3.626 -> 6.436 | Helped |
| Add the heat conduction constraint (ablation) | HCLFuse physics-guided diffusion · MSRS | AG | Helped |
| Add the structure preservation constraint after heat conduction (ablation) | HCLFuse physics-guided diffusion · MSRS | SCD | Helped |
| Add the physical consistency constraint to heat conduction and structure preservation (ablation) | HCLFuse physics-guided diffusion · MSRS | AG | Helped |
Surprising findings
- Several metrics are not best for HCLFuse; e.g., DDFM and Text-DiFuse achieve higher CC and SD on some datasets, and Text-DiFuse has highest SD on TNO-related table comparisons, and HCLFuse's BRISQUE on MSRS is second-best behind TarDAL.
Limitations
- Relies on well-aligned infrared and visible image pairs
- Computational overhead of the diffusion process limits real-time or resource-constrained use
Experimental setup
Datasets: MSRS TNO FMB MFNet
Benchmarks: MSRS TNO FMB MFNet
Metrics: SD AG CC SCD EN SF Nabf DF QSF VIF
Baselines: FusionGAN NestFuse SwinFusion TarDAL SegMiF SOSMaskFuse LRRNet STFNet CrossFuse DDFM Diff-IF Text-IF
Builds on: DDIM information bottleneck variational autoencoder Mask2Former
Closest papers by meaning
- Every SAM Drop Counts: Embracing Semantic Priors for Multi-Modality Image Fusion and Beyond (CVPR 2025 · similarity 0.86)
- MRFS: Mutually Reinforcing Image Fusion and Segmentation (CVPR 2024 · similarity 0.86)
- DCEvo: Discriminative Cross-Dimensional Evolutionary Learning for Infrared and Visible Image Fusion (CVPR 2025 · similarity 0.85)
- Text-DiFuse: An Interactive Multi-Modal Image Fusion Framework based on Text-modulated Diffusion Model (NeurIPS 2024 · similarity 0.85)
- Image Fusion via Vision-Language Model (ICML 2024 · similarity 0.85)
- Probing Synergistic High-Order Interaction in Infrared and Visible Image Fusion (CVPR 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.