Research map / Vision-language and multimodal / Cross-modal learning

When Integral Meets Decomposition: A Signal-Level Self-Supervised Feature Decompose Paradigm for Multi-Modal Image Fusion

NeurIPS 2026 · Vision-language and multimodal › Cross-modal learning · approach: Multi-modality image fusion · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes SIDFusion, a two-stage self-supervised multimodal image fusion method that decomposes 1D DWT-transformed feature signals into common and unique parts, supervised by an integral-area loss between common and source signals plus an image-level reconstruction loss.

Problem

Feature-decomposition-based multimodal image fusion lacks ground-truth common/unique feature supervision, so decomposers rely on hand-crafted, possibly conflicting combinations of image-level metric losses.

Approach

Reformulates feature decomposition as a 1D signal-level integral optimization: the common signal is trained to minimize the integral squared deviation from both source signals (after DWT into low/high-frequency bands), paired with an image-level reconstruction pretext task in a two-stage SSL pipeline.

Key results

• On M3FD VIF, the method attains best Q_MI (0.6608), Q_NICE (0.8209), Q_P (0.5735) and VIF_p (0.4694) among compared methods, but not best on all metrics.
• On MRI-CT MIF, Q_MI reaches 0.9369 vs 0.8403 for the next best CDDFuse-era baseline (CDDFuse 0.8403), a sizable gain.
• Plugging the signal-level decomposer into DIDFuse, CDDFuse, FD-Fuse and C2RF improves Q_MI by 14.7% to 29.6% in the appendix refinement table.

Surprising findings

• Signal-level decomposition as a plug-in improves existing decomposition-based fusion models without changing their defaults, with Q_MI gains of 14.7-29.6%.

Limitations

• Paper acknowledges DWT introduces slight additional overhead.
• Ours is not best on every metric (e.g., lower Q_CB on some sets and lower Q_Y on TNO versus DCEvo).

Experimental setup

Datasets: MSRS Harvard Medical image dataset

Benchmarks: M3FD TNO MSRS MRI-CT MRI-PET MRI-SPECT

Metrics: Q_MI Q_NICE Q_P Q_CB MI VIF_p Q_Y mAP IoU Dice

Baselines: CDDFuse LRRNet EMMA TC-MoA Text-Difuse DCEvo SAGE TDFusion Omni-Fuse C2RF SigFusion CCF

Builds on: DWT Restormer CDDFuse SigFusion

Closest papers by meaning

• SigFusion: Unified Signal-Level Self-Supervised Learning Paradigm for Image Fusion (AAAI 2026 · similarity 0.88)
• Beyond Spatial-Domain Supervision: A Relation Constrained Space for Multi-Modal Image Fusion (NeurIPS 2026 · similarity 0.85)
• GeminiFusion: Efficient Pixel-wise Multimodal Fusion for Vision Transformer (ICML 2024 · similarity 0.84)
• Hybrid-Supervised Dual-Search: Leveraging Automatic Learning for Loss-Free Multi-Exposure Image Fusion (AAAI 2024 · similarity 0.84)
• E2E-MFD: Towards End-to-End Synchronous Multimodal Fusion Detection (NeurIPS 2024 · similarity 0.84)
• One Model for ALL: Low-Level Task Interaction Is a Key to Task-Agnostic Image Fusion (CVPR 2025 · similarity 0.83)