Research map / Image generation / Editing and style transfer
DreamLight: Towards Harmonious and Consistent Image Relighting
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces DreamLight, a single diffusion-based model for image-based and text-based universal relighting of composited subjects, with a position-guided light adapter and a wavelet-based spectral foreground fixer.
Problem
Compositing a subject into a new background so that lighting and color tone are harmonized, supporting both background images and text prompts without environment maps or HDR inputs; prior methods either need intrinsic decomposition/light-stage data or produce weak light interaction and foreground d
Approach
Reorganizes inputs (masked foreground, background, noise) into a unified format on a Stable Diffusion backbone; a Position-Guided Light Adapter condenses background light from four directions into direction-specific query embeddings with direction-biased masked cross-attention, and a Spectral Foreground Fixer uses wavelet decomposition with learned modulation coefficients to restore foreground hig
Key results
- Image-based relighting: PSNR 22.15, SSIM 0.783, LPIPS 0.158, CLIP-IS 0.908, beating IC-Light (20.27 PSNR) and IH (20.66 PSNR).
- Text-based relighting: CLIP score 0.644, aesthetic score 6.32, ImageReward 3.47, versus IC-Light (0.629, 6.04, 2.25).
- SFF ablation on face crops: PSNR 25.53 vs 19.31 without fixer and 19.69 with ControlNet.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add PGLA instead of using no light adapter (ablation) | DreamLight · Image-based relighting | PSNR: 18.58 -> 22.15 | Helped |
| replace PGLA with vanilla IP-Adapter (ablation) | DreamLight · Image-based relighting | PSNR: 22.15 -> 20.23 | Hurt |
| add the spectral filter to PGLA (ablation) | DreamLight · Image-based relighting | PSNR: 21.81 -> 22.15 | Helped |
| add SFF instead of leaving the fixer out (ablation) | DreamLight · Face-region relighting | PSNR: 19.31 -> 25.53 | Helped |
| replace SFF with ControlNet for subject information (ablation) | DreamLight · Face-region relighting | PSNR: 25.53 -> 19.69 | Hurt |
Limitations
- Relies on a 600-pair synthetic rendered benchmark for image-based evaluation, not real paired data.
- Text-based evaluation uses automatic metrics and a user study only in supplementary material.
- Authors note SFF's effect on global metrics is limited, so gains are measured on cropped face regions.
Experimental setup
Datasets: time-lapse photography videos self-photographed photos 3D assets rendered with Arnold Renderer IC-Light-processed natural images
Benchmarks: 600-pair Arnold-rendered relighting evaluation benchmark
Metrics: PSNR SSIM LPIPS CLIP-IS CLIP score Aesthetic score ImageReward
Baselines: Harmonizer INR PCT IC-Light PowerPaint BrushNet ControlNet vanilla IP-Adapter
Builds on: IP-Adapter IC-Light Stable Diffusion ControlNet OminiControl
Closest papers by meaning
- Relightful Harmonization: Lighting-aware Portrait Background Replacement (CVPR 2024 · similarity 0.91)
- Comprehensive Relighting: Generalizable and Consistent Monocular Human Relighting and Harmonization (CVPR 2025 · similarity 0.90)
- Text2Relight: Creative Portrait Relighting with Text Guidance (AAAI 2025 · similarity 0.90)
- SynthLight: Portrait Relighting with Diffusion Model by Learning to Re-render Synthetic Faces (CVPR 2025 · similarity 0.89)
- UniRelight: Learning Joint Decomposition and Synthesis for Video Relighting (NeurIPS 2025 · similarity 0.89)
- UniLumos: Fast and Unified Image and Video Relighting with Physics-Plausible Feedback (NeurIPS 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.