Research map / Image generation / Image restoration
LEDiff: Latent Exposure Diffusion for HDR Generation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
LEDiff fine-tunes a pretrained Stable Diffusion VAE decoder and denoisers on a small HDR dataset and fuses latent exposure brackets in latent space, enabling HDR image generation and LDR-to-HDR conversion.
Problem
Latent diffusion models are limited to 8-bit LDR outputs, and existing LDR-to-HDR (inverse tone mapping) methods struggle to hallucinate photorealistic, physically plausible content in clipped highlights and shadows.
Approach
Keep the pretrained latent space of a diffusion model frozen in meaning, generate a latent exposure bracket (under/over-exposed latent codes) via highlight and shadow denoisers, merge them with a learnable depth-wise convolution softmax fusion module in latent space, and decode to HDR with a fine-tuned VAE decoder that linearizes and expands dynamic range.
Key results
- On highlight hallucination, LEDiff achieves FID-R 12.70 vs 13.39 for HDRCNN and best PU21-PIQE 48.46 among generalized methods.
- On shadow hallucination, LEDiff achieves PU21-PIQE 43.37 and FID-R 20.93, outperforming all baselines (e.g., HDRCNN FID-R 32.55).
- In a user study with 1200 pairwise comparisons, users preferred LEDiff over HDRCNN (84.19%), MaskHDR (88.22%), SingleHDR (89.40%), and ExpandNet (94.52%).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use LEDiff instead of HDRCNN for highlight reconstruction | LDR-to-HDR reconstruction · SI-HDR highlights | PU21-PIQE: 49.43 ± 6.91 -> 48.46 ± 7.04 | Helped |
| Use LEDiff instead of HDRCNN for shadow reconstruction | LDR-to-HDR reconstruction · SI-HDR shadow | FID-R: 32.55 -> 20.93 | Helped |
| Use LEDiff instead of SingleHDR for shadow reconstruction | LDR-to-HDR reconstruction · SI-HDR shadow | HDR-VDP3: 7.47 ± 0.78 -> 6.98 ± 0.83 | Hurt |
| Add VAE decoder fine-tuning (ablation) | LEDiff · SI-HDR highlights | HDR-VDP3: 4.67 ± 0.87 -> 6.16 ± 0.97 | Helped |
| Add VAE decoder fine-tuning (ablation) | LEDiff · SI-HDR highlights | PU21-PIQE: 48.60 ± 7.02 -> 48.46 ± 7.04 | Helped |
| Add denoiser fine-tuning (ablation) | LEDiff · SI-HDR highlights | HDR-VDP3: 5.59 ± 1.12 -> 6.16 ± 0.97 | Helped |
| Add denoiser fine-tuning (ablation) | LEDiff · SI-HDR highlights | PU21-PIQE: 50.25 ± 6.55 -> 48.46 ± 7.04 | Helped |
| Replace denoiser fine-tuning with SD-based inpainting (ablation) | LEDiff · SI-HDR highlights | HDR-VDP3: 3.65 ± 1.20 vs. 6.16 ± 0.97 | Hurt |
| Replace denoiser fine-tuning with SD-based inpainting (ablation) | LEDiff · SI-HDR highlights | PU21-PIQE: 50.18 ± 7.53 vs. 48.46 ± 7.04 | Hurt |
| Use LEDiff rather than competing methods in pairwise user comparisons | LDR-to-HDR reconstruction · HDR-display user study | user preference: 84.19%; 88.22%; 89.40%; 94.52% | Helped |
Surprising findings
- Full-reference HDR-VDP3 is competitive but not uniformly best (HDRCNN and MaskHDR score higher on highlights), motivating FID and no-reference metrics.
Limitations
- Inherits strengths and weaknesses of Stable Diffusion; video results using Stable Video Diffusion show more artifacts.
- Does not handle compression artifacts or noise in input LDR images.
- Evaluation relies on full-reference metrics poorly suited to hallucinated content, so FID and user studies are used.
Experimental setup
Datasets: HDR image collections (36,000 HDR images from 8 sources, including panorama datasets) SI-HDR
Benchmarks: SI-HDR
Metrics: HDR-VDP3 PU21-PIQE FID (tone-mapped with Reinhard, Durand, Liang) user preference study
Baselines: HDRCNN MaskHDR SingleHDR ExpandNet GlowGAN Stable Diffusion MVDiffusion
Builds on: Stable Diffusion exposure fusion HDRCNN GlowGAN
Closest papers by meaning
- Single Shot HDR Recovery via a Video Diffusion Prior (NeurIPS 2026 · similarity 0.88)
- High Dynamic Range Novel View Synthesis with Single Exposure (ICML 2025 · similarity 0.87)
- Generating Content for HDR Deghosting from Frequency View (CVPR 2024 · similarity 0.87)
- HDR-GS: Efficient High Dynamic Range Novel View Synthesis at 1000x Speed via Gaussian Splatting (NeurIPS 2024 · similarity 0.86)
- Perceptual Assessment and Optimization of HDR Image Rendering (CVPR 2024 · similarity 0.86)
- EvHDR-NeRF: Building High Dynamic Range Radiance Fields with Single Exposure Images and Events (AAAI 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.