Research map / Image generation / Image restoration
ISPDiffuser: Learning RAW-to-sRGB Mappings with Texture-Aware Diffusion Models and Histogram-Guided Color Consistency
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes ISPDiffuser, a diffusion-based decoupled RAW-to-sRGB framework that reconstructs grayscale texture with a texture-aware diffusion model and maps grayscale to color with a histogram-guided color consistency module.
Problem
Learning-based RAW-to-sRGB (ISP emulation) methods suffer from local detail disparity and global color distortion, and existing diffusion approach DiffRAW depends on LiteISPNet outputs as conditions.
Approach
Decouple RAW-to-sRGB into grayscale detail reconstruction via a diffusion model with a Canny-based texture enrichment loss, followed by color histogram-guided colorization of the grayscale feature with a color consistency loss.
Key results
- On ZRR (Align GT with RAW), ISPDiffuser reaches 24.09 dB PSNR, 0.881 SSIM, 0.111 LPIPS, beating FourierISP (23.93 dB).
- On MAI, ISPDiffuser achieves 25.64 dB PSNR, 0.894 SSIM, 0.071 LPIPS, the best across all three metrics.
- Ablation: replacing HCCM with ColorFormer or DDColor lowers PSNR (19.24 and 18.83 dB) versus HCCM (20.93 dB) in the decoupled setting.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| decouple grayscale detail reconstruction from color mapping (ablation) | ISPDiffuser · ZRR test set, Original GT | PSNR: 20.12 -> 20.93 | Helped |
| replace HCCM with DDColor (ablation) | ISPDiffuser · ZRR test set, Original GT | PSNR: 18.83 vs 20.93 | Hurt |
| replace HCCM with ColorFormer (ablation) | ISPDiffuser · ZRR test set, Original GT | PSNR: 19.24 vs 20.93 | Hurt |
| add texture enrichment loss (ablation) | ISPDiffuser · ZRR test set, Original GT | PSNR: 21.30 -> 21.33 | Helped |
| add texture enrichment loss (ablation) | ISPDiffuser · ZRR test set, Original GT | SSIM: 0.736 -> 0.751 | Helped |
| add texture enrichment loss (ablation) | ISPDiffuser · ZRR test set, Original GT | LPIPS: 0.161 -> 0.156 | Helped |
| add color consistency loss (ablation) | ISPDiffuser · ZRR test set, Original GT | PSNR: 21.30 -> 21.62 | Helped |
| add color consistency loss (ablation) | ISPDiffuser · ZRR test set, Original GT | SSIM: 0.736 -> 0.750 | Helped |
| add color consistency loss (ablation) | ISPDiffuser · ZRR test set, Original GT | LPIPS: 0.161 -> 0.158 | Helped |
| use ISPDiffuser instead of FourierISP | RAW-to-sRGB mapping · ZRR test set, Original GT | PSNR: 21.65 -> 21.77 | Helped |
| use ISPDiffuser instead of FourierISP | RAW-to-sRGB mapping · ZRR test set, Original GT | SSIM: 0.755 -> 0.754 | Hurt |
| use ISPDiffuser instead of FourierISP | RAW-to-sRGB mapping · ZRR test set, Align GT with RAW | PSNR: 23.93 -> 24.09 | Helped |
Surprising findings
- Decoupled ablation shows the diffusion baseline without decoupling gets 20.12 dB, and generic colorizers (DDColor, ColorFormer) in the decoupled framework underperform the no-decouple baseline.
Limitations
- Generalization limited to training data from specific cameras, weather and lighting conditions.
- Diffusion inference with 25 sampling steps is much slower than lightweight CNN methods (490 ms vs ~23-25 ms).
- Original-GT SSIM and LPIPS are slightly inferior to FourierISP and DiffRAW on ZRR due to misalignment.
Experimental setup
Datasets: ZRR MAI
Benchmarks: ZRR MAI
Metrics: PSNR SSIM LPIPS MUSIQ TOPIQ
Baselines: PyNet AWNet MW-ISPNet LiteISPNet FourierISP DiffRAW ColorFormer DDColor
Builds on: DiffRAW LiteISPNet denoising diffusion probabilistic models DDIM
Closest papers by meaning
- DiffRAW: Leveraging Diffusion Model to Generate DSLR-Comparable Perceptual Quality sRGB from Smartphone RAW Images (AAAI 2024 · similarity 0.90)
- Enhancing RAW-to-sRGB with Decoupled Style Structure in Fourier Domain (AAAI 2024 · similarity 0.86)
- PixelART: Image-to-Layer Decomposition without Latents or Text-to-Image Pretraining (NeurIPS 2026 · similarity 0.84)
- Color Alignment in Diffusion (CVPR 2025 · similarity 0.84)
- GCC: Generative Color Constancy via Diffusing a Color Checker (CVPR 2025 · similarity 0.84)
- RGB-Event ISP: The Dataset and Benchmark (ICLR 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.