Research map / Image generation / Editing and style transfer
DiffRetouch: Using Diffusion to Retouch on the Shoulder of Experts
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes DiffRetouch, a Stable Diffusion-based image retouching model that samples diverse expert-style results from four adjustable attributes, using an affine bilateral grid and a contrastive loss.
Problem
Deterministic retouching models average over expert styles and produce only one output per input, and existing style-controllable methods need reference images to specify the target style.
Approach
Use a conditional latent diffusion model (Stable Diffusion 2.1-base) whose condition vector holds four adjustable attribute coefficients (colorfulness, contrast, color temperature, brightness), with an affine bilateral grid predicted per denoising step to avoid texture distortion and an InfoNCE-style contrastive loss to make the attributes controllable.
Key results
- On MIT-Adobe FiveK with five expert subsets, DiffRetouch (single model) reaches 25.20 PSNR / 0.946 SSIM average and FID 8.957, versus 24.09 / 0.928 and 9.493 for StarEnhancer.
- On FiveK Expert-C at 340p, DiffRetouch achieves 26.21 PSNR and 0.944 SSIM, above TSFlow (25.57 / 0.935) and StarEnhancer (25.73 / 0.937).
- Ablation on FiveK: the affine bilateral grid and contrastive loss together reduce FID from 26.721 (CFW variant) to 8.957 for the full model.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add the affine bilateral grid to the baseline (ablation) | DiffRetouch baseline · Adobe5K | PSNR: 19.83 -> 23.70 | Helped |
| Add the affine bilateral grid to the baseline (ablation) | DiffRetouch baseline · PPR10K | PSNR: 20.47 -> 25.15 | Helped |
| Add CFW to the baseline (ablation) | DiffRetouch baseline · Adobe5K | PSNR: 19.83 -> 20.97 | Helped |
| Add contrastive learning to the affine-bilateral-grid model (ablation) | DiffRetouch · Adobe5K | FID: 11.476 -> 8.957 | Helped |
| Add contrastive learning to the affine-bilateral-grid model (ablation) | DiffRetouch · PPR10K | FID: 3.091 -> 2.499 | Helped |
| Add contrastive learning (ablation) | DiffRetouch with affine bilateral grid · Adobe5K attribute control range | Colorfulness range: 10.12 -> 20.60 | Helped |
| Add contrastive learning (ablation) | DiffRetouch with affine bilateral grid · Adobe5K attribute control range | Contrast range: 252.7 -> 297.3 | Helped |
| Add contrastive learning (ablation) | DiffRetouch with affine bilateral grid · Adobe5K attribute control range | Color temperature range: 1141.5 -> 1572.9 | Helped |
| Add contrastive learning (ablation) | DiffRetouch with affine bilateral grid · Adobe5K attribute control range | Brightness range: 26.46 -> 35.22 | Helped |
| Use DiffRetouch instead of StarEnhancer | MIT-Adobe FiveK mixture of experts A/B/C/D/E · MIT-Adobe FiveK | Average PSNR: 24.09 -> 25.20 | Helped |
| Use DiffRetouch instead of RSFNet | PPR10K mixture of experts A/B/C · PPR10K | FID: 2.467 -> 2.477 | Hurt |
| Use 20 sampling steps rather than fewer steps (ablation) | DiffRetouch inference · Image retouching | PSNR | Helped |
Surprising findings
- Deterministic retouching baselines can be outperformed in PSNR by a single multi-style model trained on mixed expert subsets.
- Adjustable range of attribute scores is much larger with contrastive learning, e.g. color temperature range rises from 1141.5 to 1572.9 on FiveK.
Limitations
- Fails on extreme cases where the desired expert style lies at the edge of the adjustable range; users must push coefficients beyond [-1,1].
- Relies on hand-crafted attribute metrics (colorfulness, contrast, CCT, brightness) to build condition labels from expert GT.
Experimental setup
Datasets: MIT-Adobe FiveK PPR10K
Benchmarks: MIT-Adobe FiveK PPR10K
Metrics: PSNR SSIM LPIPS FID NIMA
Baselines: HDRNet DeepUPE CURL DeepLPF 3DLUT CSRNet RSFNet PIENet TSFlow StarEnhancer
Builds on: Stable Diffusion DDPM HDRNet StableSR InfoNCE
Closest papers by meaning
- High-Fidelity Diffusion-Based Image Editing (AAAI 2024 · similarity 0.86)
- PAIR Diffusion: A Comprehensive Multimodal Object-Level Image Editor (CVPR 2024 · similarity 0.86)
- DragonDiffusion: Enabling Drag-style Manipulation on Diffusion Models (ICLR 2024 · similarity 0.86)
- PHR-DIFF: Portrait Highlights Removal via Patch-aware Diffusion Model (AAAI 2025 · similarity 0.86)
- Diffusing Differentiable Representations (NeurIPS 2024 · similarity 0.85)
- DiffRAW: Leveraging Diffusion Model to Generate DSLR-Comparable Perceptual Quality sRGB from Smartphone RAW Images (AAAI 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.