Research map / Vision-language and multimodal / Multimodal LLMs

Robust Fusion Controller: Degradation-Aware Image Fusion with Fine-Grained Language Instructions

AAAI 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Interleaved image-text generation with MLLMs · 1 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes RFC, a language-instruction-controlled image fusion network that parses instructions into functional and spatial conditions, couples them into a control prior that modulates hybrid-attention fusion blocks, and adds a language-feature alignment loss to remove spatially varying composite degradations.

Problem

Image fusion in real adverse environments suffers from diverse, composite, spatially varying degradations (low light, flare, haze, noise, blur), which existing fusion methods neither remove nor handle with fine-grained control.

Approach

Language instructions are parsed into a functional condition (CLIP text embedding of the degradation type) and a spatial condition (CLIPSeg region maps, partly fine-tuned on multi-modal data); these are coupled via FiLM-style affine modulation into a composite control prior that modulates HAF fusion blocks, trained with a degradation-aware reconstruction loss and a language-feature alignment loss.

Key results

• On composite-degradation test sets, RFC reports SSIM 0.452 on M3FD and 0.400 Qabf on MFNet, with best scores on most metrics versus nine SOTA fusion methods.
• On LLVIP object detection with retrained YOLO-v5, RFC reaches mAP@(0.5:0.95) of 75.3, above the next best MRFS at 74.4.
• Ablation on the mean of four datasets shows the full model achieves SSIM 0.325 and Qabf 0.469, the best among ablation variants.

Surprising findings

• MRFS reaches higher semantic segmentation mIoU in some categories than RFC, and RFC is second only to MRFS on the FMB segmentation evaluation (per the paper's own explanation).

Limitations

• Evaluation relies on simulated degradations rather than real-world adverse captures.
• Baselines without degradation removal are pre-processed with InstructIR, which may affect fairness of comparison.
• Degradation-free and generalization results are mixed; RFC does not win every metric (e.g., SCD on MFNet and MI on M2VD).

Experimental setup

Datasets: MFNet LLVIP M3FD FMB RoadScene M2VD

Benchmarks: M3FD MFNet LLVIP FMB M2VD

Metrics: Qabf SSIM SD MI VIF SCD AG EN SF precision

Baselines: MRFS CDDFuse DDFM CAF SHIP FISCNet ReFusion SDCFusion Text-IF

Builds on: FiLM CLIPSeg CBAM-style hybrid attention Text-IF InstructIR

Closest papers by meaning

• ControlFusion: A Controllable Image Fusion Network with Language-Vision Degradation Prompts (NeurIPS 2025 · similarity 0.92)
• Task-Gated Multi-Expert Collaboration Network for Degraded Multi-Modal Image Fusion (ICML 2025 · similarity 0.85)
• Text-IF: Leveraging Semantic Text Guidance for Degradation-Aware and Interactive Image Fusion (CVPR 2024 · similarity 0.85)
• Text-DiFuse: An Interactive Multi-Modal Image Fusion Framework based on Text-modulated Diffusion Model (NeurIPS 2024 · similarity 0.84)
• Multi-axis Prompt and Multi-dimension Fusion Network for All-in-one Weather-degraded Image Restoration (AAAI 2025 · similarity 0.84)
• Conditional Controllable Image Fusion (NeurIPS 2024 · similarity 0.83)