Research map / Detection and segmentation / CNN features
Task-driven Image Fusion with Learnable Fusion Loss
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes TDFusion, a meta-learning framework where a loss generation module produces learnable intensity weights for an infrared-visible fusion loss, trained via the downstream task loss (segmentation/detection) through MAML-style inner/outer updates.
Problem
Multi-modal image fusion methods for downstream tasks use predefined fusion loss terms that may mismatch the task objectives, limiting adaptivity and flexibility.
Approach
A loss generation module outputs per-pixel softmax intensity weights for the fusion loss; it is optimized by backpropagating the downstream task loss through a one-step updated fusion network (MAML-style meta-learning), so the fusion loss becomes task-driven and independent of task or architecture.
Key results
- On FMB, TDFusion reaches mIoU 75.09 on MSRS segmentation and 60.50 mIoU on FMB, outperforming SegMIF (74.25 / 58.41).
- Object detection on M3FD achieves mAP50 86.27 versus 83.71 for the best prior baseline EMMA.
- Ablation on FMB shows removing learnable loss (fixed weights 1/2) drops SF from 14.16 to 13.73 and VIF from 0.43 to 0.39.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use TDFusion instead of MRFS | semantic segmentation on MSRS · MSRS | mIoU: 74.50 -> 75.09 | Helped |
| Use TDFusion instead of SegMIF | semantic segmentation on FMB · FMB | mIoU: 58.41 -> 60.50 | Helped |
| Use TDFusion instead of EMMA | object detection on M3FD · M3FD | mAP50: 83.71 -> 86.27 | Helped |
| Use TDFusion instead of DCINN | object detection on LLVIP · LLVIP | AP50: 94.92 -> 95.00 | Helped |
| Use TDFusion instead of EMMA | infrared-visible fusion on MSRS · MSRS | SCD: 1.62 -> 1.86 | Helped |
| Use TDFusion instead of SegMIF | infrared-visible fusion on FMB · FMB | SCD: 1.72 -> 1.76 | Helped |
| Use TDFusion instead of SegMIF | infrared-visible fusion on M3FD · M3FD | SCD: 1.72 -> 1.83 | Helped |
| Use TDFusion instead of EMMA | infrared-visible fusion on LLVIP · LLVIP | SCD: 1.57 -> 1.75 | Helped |
| Add learnable intensity weights instead of fixing wa and wb to 1/2 (ablation) | TDFusion on FMB · FMB | SCD: 1.58 -> 1.76 | Helped |
| Add the gradient loss (ablation) | TDFusion on FMB · FMB | SCD: 1.63 -> 1.76 | Helped |
| Keep task loss from directly influencing fusion module parameters (ablation) | TDFusion on FMB · FMB | SCD: 1.70 -> 1.76 | Helped |
| Add a dedicated fusion-module learning phase (ablation) | TDFusion on FMB · FMB | SCD: 1.72 -> 1.76 | Helped |
Limitations
- Relies on specific lightweight downstream networks (SegFormer, YOLOv8) during meta-learning; generality beyond these tasks is not fully established.
- Custom train/test splits for M3FD and LLVIP derived by the authors, complicating direct comparison.
Experimental setup
Datasets: MSRS FMB M3FD LLVIP
Benchmarks: MSRS FMB M3FD LLVIP
Metrics: EN SF SCD VIF QAB/F SSIM mAcc mIoU mAP50 mAP75
Baselines: TarDAL SegMIF MURF EMMA DCINN MRFS TIMFusion
Builds on: MAML Restormer SegFormer YOLOv8 TarDAL
Closest papers by meaning
- Efficient Rectified Flow for Image Fusion (NeurIPS 2025 · similarity 0.87)
- Task-Gated Multi-Expert Collaboration Network for Degraded Multi-Modal Image Fusion (ICML 2025 · similarity 0.86)
- ControlFusion: A Controllable Image Fusion Network with Language-Vision Degradation Prompts (NeurIPS 2025 · similarity 0.85)
- One Model for ALL: Low-Level Task Interaction Is a Key to Task-Agnostic Image Fusion (CVPR 2025 · similarity 0.85)
- Task-Customized Mixture of Adapters for General Image Fusion (CVPR 2024 · similarity 0.85)
- Every SAM Drop Counts: Embracing Semantic Priors for Multi-Modality Image Fusion and Beyond (CVPR 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.