Research map / Image generation / Image restoration
When Shadow Removal Meets Intrinsic Image Decomposition: A Joint Learning Framework Using Unpaired Data
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Jointly learns intrinsic image decomposition and shadow removal from unpaired shadow and shadow-free images, removing shadows by recovering shadow-free illumination and recombining it with the estimated reflectance.
Problem
Shadow removal typically needs paired shadow/shadow-free data or masks; unpaired methods leave shadow residuals and appearance artifacts, and intrinsic image decomposition is rarely used due to ambiguity and scarce training data.
Approach
Decompose the shadow image into reflectance and illumination with an IID network, then recover shadow-free illumination via attention-weighted fusion of illumination compensation images, and recombine with reflectance; reflectance-consistency losses computed on shadow-generated images, trained with cycle-GAN-style unpaired losses and no shadow masks at test time.
Key results
- On ISTD+, achieves 4.13 overall MAE (7.06 shadow) among unpaired methods, outperforming MaskGAN, DC-Net, LG-Net, and comparable to supervised methods.
- On IIW, reaches 26.3% WHDR for intrinsic decomposition, beating most IID baselines except UID (18.2%, trained on IIW).
- Ablation: removing IID-Net raises shadow-region MAE from 7.06 to 10.51 on ISTD+.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use the joint IID and illumination-recovery framework instead of LG-Net | Unpaired shadow removal · ISTD+ | MAE (All): 4.52 -> 4.13 | Helped |
| Use the joint IID and illumination-recovery framework instead of DC-Net | Unpaired shadow removal · SRD | MAE (All): 6.43 -> 5.61 | Helped |
| Use the joint framework instead of UID | Intrinsic image decomposition · IIW | WHDR: 18.2% -> 26.3% | Hurt |
| Add IID-Net and recover illumination in decomposed space (ablation) | Ours · ISTD+ | MAE (Shadow): 10.51 -> 7.06 | Helped |
| Use IR-Net instead of the generator in Hu et al. (2019b) (ablation) | Ours · ISTD+ | MAE (Non-shadow): 4.41 -> 3.65 | Helped |
| Add L1_rc reflectance consistency loss (ablation) | Ours · ISTD+ | MAE (All): 4.37 -> 4.13 | Helped |
| Add L2_rc reflectance consistency loss (ablation) | Ours · ISTD+ | MAE (All): 4.35 -> 4.13 | Helped |
| Use a test-time mask | Ours · ISTD+ | MAE (All): 4.13 -> 3.21 | Helped |
Surprising findings
- Naively learning illumination residuals or using exposure fusion from prior work performs poorly within this framework.
- Trained only on SRD, the IID model outperforms specialized IID methods on outdoor shadow images, and the shadow removal model generalizes to document images.
Limitations
- Shadow-region gains over supervised methods are limited; non-shadow performance benefits from test-time masks used by BADC.
- Relies on Otsu-derived masks during training for shadow generation.
Experimental setup
Datasets: ISTD+ SRD USR SBU WSRD VSR IIW NED
Benchmarks: ISTD+ SRD USR SBU IIW
Metrics: MAE in LAB color space (shadow, non-shadow, all) WHDR IOU BER
Baselines: DeshadowNet ST-CGAN DHAN Auto-Exposure Inpaint4shadow ShadowFormer ShadowDiff RRL-Net HomoFormer LFG-Diffusion DeS3 MaskGAN
Builds on: MaskGAN CycleGAN Auto-Exposure shadow removal BADC
Closest papers by meaning
- Recasting Regional Lighting for Shadow Removal (AAAI 2024 · similarity 0.88)
- OmniSR: Shadow Removal Under Direct and Indirect Lighting (AAAI 2025 · similarity 0.88)
- Polarization Guided Mask-Free Shadow Removal (AAAI 2025 · similarity 0.88)
- Detail-Preserving Latent Diffusion for Stable Shadow Removal (CVPR 2025 · similarity 0.87)
- SoftShadow: Leveraging Soft Masks for Penumbra-Aware Shadow Removal (CVPR 2025 · similarity 0.86)
- MetaShadow: Object-Centered Shadow Detection, Removal, and Synthesis (CVPR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.