Research map / Vision-language and multimodal / Cross-modal learning
Dispel Darkness for Better Fusion: A Controllable Visual Enhancer based on Cross-modal Conditional Adversarial Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes DDBF, a controllable visual enhancer that uses cross-modal conditional adversarial learning to brighten low-light visible images guided by infrared contrast, then fuses them with infrared images using explicit information preservation and hue fidelity losses.
Problem
Infrared and visible image fusion at night suffers from weakened visible information in low light, causing information mismatch, texture/contrast loss from multi-objective weighting, and color distortion from the YCbCr color separation strategy.
Approach
Uses infrared images' light-invariant contrast as the target illumination distribution for a CGAN whose condition is an enhancement ratio (built from gamma-augmented infrared samples), enabling controllable low-light enhancement without reference images; the fusion module replaces weighted multi-modal losses with max-based explicit preservation losses and a cosine-based hue fidelity loss.
Key results
- On AGLIE, Ours (r=1.8) reaches SSIM 0.675-0.706 and PSNR up to 16.434 (r=2.0 gives 16.427), outperforming BPDHE, SRIE, RetinexDIP, RUAS and SCI in PSNR/SSIM.
- On LLVIP, Ours (r=1.0) achieves MI 2.976, VIF 0.473, AG 8.020, SD 0.189 and ranks first on all four fusion metrics against DenseFuse, IFCNN, RFN-Nest, U2Fusion and SDNet.
- For high-level tasks, retrained YOLOv5 on the r=1.5 fused images achieves mAP@0.5 of 0.934 vs 0.927 for VIS+IR, and SegNeXt mIOU of 41.628 vs 40.956 for VIS+IR.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| remove illumination adversarial loss (ablation) | DDBF GRM · AGLIE | SSIM: 0.706 -> 0.405 | Hurt |
| remove low-pass filtering (ablation) | DDBF GRM · AGLIE | SSIM: 0.706 -> 0.123 | Hurt |
| replace CGAN with conventional GAN (ablation) | DDBF GRM · AGLIE | SSIM: 0.706 -> 0.526 | Hurt |
| remove hue fidelity loss (ablation) | DDBF IFM · LLVIP | VIF: 0.499 -> 0.483 | Hurt |
| replace significant contrast loss with multi-modal weighted-game loss (ablation) | DDBF IFM · LLVIP | AG: 9.150 -> 6.165 | Hurt |
| replace sharpened texture loss with multi-modal weighted-game loss (ablation) | DDBF IFM · LLVIP | MI: 2.904 -> 2.773 | Hurt |
| use YCbCr color separation strategy (ablation) | DDBF IFM · LLVIP | VIF: 0.499 -> 0.408 | Hurt |
| replace maximum integration with mean integration (ablation) | DDBF IFM · LLVIP | MI: 2.904 -> 2.904 | No clear effect |
| use DDBF rather than the strongest listed baseline | Infrared-visible fusion · RoadScene | MI: 2.467 -> 3.288 | Helped |
| use DDBF rather than the strongest listed baseline | Infrared-visible fusion · RoadScene | AG: 6.708 -> 5.451 | Hurt |
Surprising findings
- Retrained detectors on fused images outperform VIS+IR input, which the authors attribute to scene information restoration.
Limitations
- Training data limited to 400 manually selected LLVIP pairs due to imperfect registration in the full dataset.
- Fusion metrics on some daylight RoadScene settings are mixed (e.g., r=0.4 has lower VIF/AG than some baselines).
Experimental setup
Datasets: LLVIP ExDark AGLIE MFNet RoadScene
Benchmarks: ExDark AGLIE LLVIP MFNet RoadScene
Metrics: NIQE PIQE SSIM PSNR MI VIF AG SD precision recall
Baselines: BPDHE SRIE RetinexDIP RUAS SCI DenseFuse IFCNN RFN-Nest U2Fusion SDNet
Builds on: conditional GAN Retinex theory explicit information preservation loss
Closest papers by meaning
- Improving Visual and Downstream Performance of Low-Light Enhancer with Vision Foundation Models Collaboration (CVPR 2025 · similarity 0.86)
- DCEvo: Discriminative Cross-Dimensional Evolutionary Learning for Infrared and Visible Image Fusion (CVPR 2025 · similarity 0.85)
- A²RNet: Adversarial Attack Resilient Network for Robust Infrared and Visible Image Fusion (AAAI 2025 · similarity 0.84)
- Zero-Reference Low-Light Enhancement via Physical Quadruple Priors (CVPR 2024 · similarity 0.84)
- Enhancing Infrared Vision: Progressive Prompt Fusion Network and Benchmark (NeurIPS 2025 · similarity 0.84)
- Specularity Factorization for Low-Light Enhancement (CVPR 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.