Research map / Image generation / Editing and style transfer
ScribbleLight: Single Image Indoor Relighting with Scribbles
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Scribble-controlled single-image indoor relighting using an albedo-conditioned Stable Diffusion v2 and a scribble/normal encoder-decoder ControlNet, trained on LSUN Bedrooms.
Problem
Relighting a single indoor room photo with precise local lighting control is hard because indoor scenes have multiple lights, clutter, and varied materials; prior generative relighting offers only coarse global control.
Approach
Condition a Stable Diffusion v2 model on the albedo image (with fixed-level noise added to albedo latents) to preserve color and texture, and add a ControlNet driven by a latent encoding of user scribbles and normals, regularized by a decoder that reconstructs normals and shading.
Key results
- ScribbleLight achieves RMSE 0.206 and LPIPS 0.394 (best-of-5 0.190/0.370) versus RGB↔X 0.269 and LightIt* 0.341 with auto-generated scribbles.
- In a user study over 30 examples and 30 participants each, 95.6% preferred ScribbleLight over RGB↔X.
- Against IC-Light with crafted prompts, ScribbleLight achieves RMSE 0.210 vs 0.278 and PSNR 14.21 vs 11.46.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use ScribbleLight instead of RGB↔X and LightIt* | Indoor relighting from auto-generated scribbles · BigTime test | RMSE: 0.269 -> 0.206; 0.341 -> 0.206 | Helped |
| Use ScribbleLight instead of RGB↔X and LightIt* | Indoor relighting from auto-generated scribbles · BigTime test | PSNR: 12.47 -> 14.29; 9.61 -> 14.29 | Helped |
| Use ScribbleLight instead of RGB↔X and LightIt* | Indoor relighting from auto-generated scribbles · BigTime test | SSIM: 0.416 -> 0.436; 0.232 -> 0.436 | Helped |
| Use ScribbleLight instead of RGB↔X and LightIt* | Indoor relighting from auto-generated scribbles · BigTime test | LPIPS: 0.439 -> 0.394; 0.564 -> 0.394 | Helped |
| Use ScribbleLight rather than IC-Light | Indoor relighting with auto-generated scribbles · 50 test images | RMSE: 0.278 -> 0.210 | Helped |
| Use ScribbleLight rather than IC-Light | Indoor relighting with auto-generated scribbles · 50 test images | PSNR: 11.46 -> 14.21 | Helped |
| Use ScribbleLight rather than IC-Light | Indoor relighting with auto-generated scribbles · 50 test images | SSIM: 0.357 -> 0.443 | Helped |
| Use ScribbleLight rather than IC-Light | Indoor relighting with auto-generated scribbles · 50 test images | LPIPS: 0.505 -> 0.397 | Helped |
| Condition Stable Diffusion on albedo rather than injecting albedo through ControlNet (ablation) | ScribbleLight · BigTime test | RMSE: 0.2305 -> 0.2059 | Helped |
| Add noise to the albedo latent (ablation) | Albedo-conditioned Stable Diffusion · BigTime test | LPIPS: 0.4193 -> 0.3942 | Helped |
| Add the normal map input (ablation) | ScribbleLight ControlNet · BigTime test | RMSE: 0.2224 -> 0.2059 | Helped |
| Add the control decoder (ablation) | ScribbleLight ControlNet · BigTime test | RMSE: 0.2098 -> 0.2059 | Helped |
Surprising findings
- Adding fixed noise to the albedo latent improves albedo preservation and lighting variation rather than degrading it.
Limitations
- Struggles with physically inconsistent scribbles, producing realistic but implausible lighting.
- Does not support colored lighting adjustments; outputs biased toward common colors like yellow and blue.
- Evaluation on a relatively small test set of 206 pairs, with user scribbles evaluated only qualitatively.
Experimental setup
Datasets: LSUN Bedrooms BigTime time-lapse dataset
Benchmarks: BigTime time-lapse test pairs
Metrics: RMSE PSNR SSIM LPIPS user preference
Baselines: LightIt* RGB↔X IC-Light
Builds on: Stable Diffusion v2 ControlNet LightIt RGB↔X DSINE
Closest papers by meaning
- LumiNet: Latent Intrinsics Meets Diffusion Models for Indoor Scene Relighting (CVPR 2025 · similarity 0.89)
- StyLitGAN: Image-Based Relighting via Latent Control (CVPR 2024 · similarity 0.87)
- Do Image Editing Models Understand Lighting? (NeurIPS 2026 · similarity 0.86)
- LightIt: Illumination Modeling and Control for Diffusion Models (CVPR 2024 · similarity 0.85)
- LightOctree: Lightweight 3D Spatially-Coherent Indoor Lighting Estimation (CVPR 2024 · similarity 0.85)
- Text2Relight: Creative Portrait Relighting with Text Guidance (AAAI 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.