Research map / Vision-language and multimodal / Cross-modal learning
Towards Unified Semantic and Controllable Image Fusion: A Diffusion Transformer Approach
Jiayang Li, Chengjie Jiang, Junjun Jiang, Pengwei Liang, Jiayi Ma, Liqiang Nie
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add noise-mask and Gaussian-noise degradations to blur-only M3 (ablation) | DiTFuse · MSRS (IVIF) | PSNR: 65.58 -> 66.63 | Improved |
| add blur and Gaussian-noise degradations to noise-mask-only M3 (ablation) | DiTFuse · MSRS (IVIF) | PSNR: 65.48 -> 66.63 | Improved |
| add blur and noise-mask degradations to Gaussian-noise-only M3 (classical MIM) (ablation) | DiTFuse · MSRS (IVIF) | CLIP-IQA+: 0.257 -> 0.392 | Improved |
| add blur and noise-mask degradations to Gaussian-noise-only M3 (classical MIM) (ablation) | DiTFuse · MSRS (IVIF) | MUSIQ: 31.09 -> 36.28 | Improved |
| add explicit task and subtask tags to training instructions (ablation) | DiTFuse · fusion and control qualitative | color fidelity and subtask controllability | Improved |
| combine M3 data with fusion data (vs fusion-only) (ablation) | DiTFuse · IVIF qualitative | visual quality | Improved |
| add fusion data to M3-only training (ablation) | DiTFuse · IVIF qualitative | artifacts | Improved |
| keep task tag at test time (vs removing it) (ablation) | DiTFuse · brightness control qualitative | brightness enhancement intensity | Improved |
| use standardized prompt template (vs removing template / noisy prompts) (ablation) | DiTFuse · noisy prompt qualitative | fusion quality and controllability | Improved |
| DiTFuse text-controlled segmentation vs LISA on fused image | DiTFuse · FMB segmentation | overall mIoU: 0.4386 -> 0.4760 | Improved |
| DiTFuse vs LISA on road class by GPT-4o precision ratio | DiTFuse · M3FD (GPT-4o seg eval) | P.Ratio: 0.25 -> 0.84 | Improved |
| DiTFuse vs LISA on person class by GPT-4o precision ratio | DiTFuse · SICE (GPT-4o seg eval) | P.Ratio: 0.12 -> 0.43 | Improved |
Closest papers by meaning
- Text-DiFuse: An Interactive Multi-Modal Image Fusion Framework based on Text-modulated Diffusion Model (NeurIPS 2024 · similarity 0.90)
- Every SAM Drop Counts: Embracing Semantic Priors for Multi-Modality Image Fusion and Beyond (CVPR 2025 · similarity 0.89)
- ControlFusion: A Controllable Image Fusion Network with Language-Vision Degradation Prompts (NeurIPS 2025 · similarity 0.88)
- Image Fusion via Vision-Language Model (ICML 2024 · similarity 0.88)
- MRFS: Mutually Reinforcing Image Fusion and Segmentation (CVPR 2024 · similarity 0.88)
- Hybrid-Supervised Dual-Search: Leveraging Automatic Learning for Loss-Free Multi-Exposure Image Fusion (AAAI 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
