Research map / Image generation / Image restoration
Conditional Controllable Image Fusion
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes CCF, a training-free controllable image fusion framework that injects a bank of fusion constraints as conditions into a pre-trained DDPM and selects them per denoising step via a sampling-adaptive gate.
Problem
Existing image fusion methods are tailored to specific scenes with fixed fusion constraints and require training, so they adapt poorly to varying and rapidly changing environments.
Approach
Inject per-sample fusion constraints (basic, enhanced, task-specific) into a pre-trained unconditional DDPM's reverse sampling as gradient-based conditions, and dynamically select the top-k enhanced conditions at each denoising step using a sampling-adaptive condition selection (SCS) gate.
Key results
- On LLVIP, CCF reports SSIM 1.22, MSE 1694, CC 0.705 (best on these), outperforming the second-best by 0.02 SSIM and 0.035 CC.
- On MFFW multi-focus, CCF achieves SD 79.02, AG 8.18, SF 24.30, SCD 2.78, the best on all four metrics.
- With a YOLOv5 detection condition on 287 LLVIP images, mAP@.5 rises from 0.858 to 0.907 (reported in text as 0.737 to 0.907) and recall improves to 0.832.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use CCF instead of the strongest listed baseline | image fusion · LLVIP | SSIM: 1.20 -> 1.22 | Helped |
| Use CCF instead of the strongest listed baseline | image fusion · LLVIP | MSE: 2056 -> 1694 | Helped |
| Use CCF instead of the strongest listed baseline | image fusion · LLVIP | CC: 0.670 -> 0.705 | Helped |
| Use CCF instead of the strongest listed baseline | image fusion · LLVIP | PSNR: 36.10 -> 32.71 | Hurt |
| Use CCF instead of the strongest listed baseline | image fusion · LLVIP | Nabf: 0.004 -> 0.005 | Hurt |
| Add basic conditions to DDPM (ablation) | CCF · LLVIP | SSIM: 0.28 -> 1.16 | Helped |
| Add enhanced conditions without SCS (ablation) | CCF with basic conditions · LLVIP | PSNR: 33.06 -> 32.17 | Hurt |
| Add enhanced conditions without SCS (ablation) | CCF with basic conditions · LLVIP | Nabf: 0.032 -> 0.072 | Hurt |
| Add SCS to basic and enhanced conditions (ablation) | CCF without SCS · LLVIP | SSIM: 1.16 -> 1.22 | Helped |
| Add SCS to basic and enhanced conditions (ablation) | CCF without SCS · LLVIP | MSE: 1779 -> 1694 | Helped |
| Increase enhanced conditions from 4 to 8 (ablation) | CCF · LLVIP | SSIM: 1.12 -> 1.22 | Helped |
| Increase enhanced conditions from 8 to 12 (ablation) | CCF · LLVIP | SSIM: 1.22 -> 1.20 | Hurt |
Surprising findings
- Adding all enhanced conditions without the SCS selection lowers several metrics versus fewer conditions, attributed to conflicts between conditions.
Limitations
- Condition bank is constructed empirically and basic vs enhanced categorization is manual.
- Reliance on a pre-trained diffusion model makes generation slow and inefficient.
- Selected conditions may not be effective across all scenarios, risky in high-stakes settings such as medical imaging and autonomous driving.
Experimental setup
Datasets: LLVIP MFFW MEFB TNO Harvard medical image dataset
Benchmarks: LLVIP MFFW MEFB TNO Harvard medical image dataset
Metrics: SSIM MSE CC PSNR Nabf SD AG SF SCD EN
Baselines: SwinFusion DIVFusion MUFusion CDDFuse DDFM Text-IF TC-MoA U2Fusion DeFusion DenseFuse RFN-Nest UMF-CMGR
Builds on: DDPM classifier guidance DDFM diffusion posterior sampling
Closest papers by meaning
- A Simple Approach to Unifying Diffusion-based Conditional Generation (ICLR 2025 · similarity 0.88)
- CCS: Controllable and Constrained Sampling with Diffusion Models via Initial Noise Perturbation (NeurIPS 2025 · similarity 0.86)
- Hybrid-Supervised Dual-Search: Leveraging Automatic Learning for Loss-Free Multi-Exposure Image Fusion (AAAI 2024 · similarity 0.85)
- Rectified Diffusion Guidance for Conditional Generation (CVPR 2025 · similarity 0.85)
- Decouple-Then-Merge: Finetune Diffusion Models as Multi-Task Learning (CVPR 2025 · similarity 0.85)
- Guided and Variance-Corrected Fusion with One-shot Style Alignment for Large-Content Image Generation (AAAI 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.