Research map / Vision-language and multimodal / Cross-modal learning
Probing Synergistic High-Order Interaction in Infrared and Visible Image Fusion
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SHIP, a paradigm that iteratively applies high-order spatial interactions (via FFT-based element-wise products) and high-order channel interactions (extending SE block statistics) for infrared-visible image fusion, with a pan-sharpening extension.
Problem
Cross-attention and self-attention in infrared-visible image fusion only capture second-order spatial interactions and ignore high-order interactions across spatial and channel dimensions, limiting exploitation of multi-modal synergies.
Approach
Replace costly matrix-multiplication attention with an equivalent element-wise Hadamard product in the frequency domain (via FFT) to extend spatial interactions to arbitrary N-order, and extend the SE block's first-order mean channel statistic to high-order channel interactions, alternated over L iterations.
Key results
- On M3FD, SHIP reports MI 4.813 and VIF 0.922, versus the best baseline SDNet MI 3.231 and DIDFuse VIF 0.877.
- On the pan-sharpening GaoFen2 dataset, SHIP surpasses the second-best INNformer by 0.105 dB PSNR (47.458 vs 47.353).
- Ablation on M3FD: performance rises with interaction order N up to 5, then stabilizes; L=3 iterations chosen as default.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use SHIP instead of the strongest listed baseline | Infrared-visible image fusion · M3FD | MI: 3.231 -> 4.813 | Helped |
| Use SHIP instead of the strongest listed baseline | Infrared-visible image fusion · RoadScene | VIF: 0.847 -> 0.905 | Helped |
| Use SHIP instead of the strongest listed baseline | Infrared-visible image fusion · TNO | VIF: 0.860 -> 0.933 | Helped |
| Increase interaction order N (ablation) | SHIP on M3FD · M3FD | Overall fusion metrics | Helped |
| Increase interaction order beyond 5 (ablation) | SHIP on M3FD · M3FD | Overall fusion metrics | No clear effect |
| Increase iteration count L (ablation) | SHIP on M3FD · M3FD | Overall fusion metrics | Helped |
| Increase iteration count further (ablation) | SHIP on M3FD · M3FD | SF | Hurt |
Surprising findings
- Performance plateaus after order N=5, and further increases in iterations L reduce SF and AG, attributed to gradient propagation challenges.
Limitations
- Higher-order interaction gains saturate beyond N=5 and deeper iterations degrade some metrics.
- Evaluation of RoadScene and TNO uses only 25 randomly selected pairs each, without predefined splits.
Experimental setup
Datasets: M3FD RoadScene TNO WorldView II GaoFen2 WorldView III
Benchmarks: M3FD RoadScene TNO WorldView II GaoFen2 WorldView III
Metrics: SF MI VIF AG Qabf FMI PSNR SSIM SAM ERGAS
Baselines: DDcGAN DenseFuse AUIF DIDFuse ReCoNet SDNet TarDAL U2Fusion UMFusion GS PANNet SRPPNN
Builds on: Transformer self-attention Squeeze-and-Excitation TarDAL SwinFusion
Closest papers by meaning
- Beyond Strict Pairing: Arbitrarily Paired Training for High-Performance Infrared and Visible Image Fusion (CVPR 2026 · similarity 0.86)
- Revisiting Spatial-Frequency Information Integration from a Hierarchical Perspective for Panchromatic and Multi-Spectral Image Fusion (CVPR 2024 · similarity 0.85)
- Revisiting Generative Infrared and Visible Image Fusion Based on Human Cognitive Laws (NeurIPS 2025 · similarity 0.84)
- DCEvo: Discriminative Cross-Dimensional Evolutionary Learning for Infrared and Visible Image Fusion (CVPR 2025 · similarity 0.83)
- Every SAM Drop Counts: Embracing Semantic Priors for Multi-Modality Image Fusion and Beyond (CVPR 2025 · similarity 0.83)
- Spectral-Spatial Interpretation (NeurIPS 2026 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.