Research map / Image generation / Image restoration
SoftShadow: Leveraging Soft Masks for Penumbra-Aware Shadow Removal
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SoftShadow, which jointly fine-tunes a LoRA-adapted SAM to predict continuous soft shadow masks and a ShadowDiffusion removal network end-to-end, guided by a penumbra formation constraint, achieving mask-free shadow removal.
Problem
Shadow removal methods rely on binary shadow masks, which cause boundary artifacts in penumbra (soft shadow) regions and depend on the quality of external mask detectors.
Approach
Replace binary shadow masks with learned soft (grayscale) shadow masks derived from a physical shadow formation model, predicted by a LoRA-tuned SAM and jointly trained with the removal network under a penumbra formation constraint loss that regularizes mask gradients in the penumbra.
Key results
- On SRD, SoftShadow reaches 35.57 dB PSNR overall, versus 35.37 dB for HomoFormer (with DHAN masks) and 34.11 dB for DeS3 without masks.
- On ISTD+, SoftShadow reaches 35.00 dB PSNR overall without input masks, versus 31.38 dB for DeS3.
- On the generalization sets, SoftShadow reaches 23.32 dB on LRSS and 28.85 dB on UIUC, outperforming the compared zero-shot and mask-free methods.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use SoftShadow instead of Homoformer | shadow removal on SRD · SRD | PSNR (all pixels): 35.37 -> 35.57 | Helped |
| use SoftShadow instead of ShadowDiffusion | shadow removal on ISTD+ · ISTD+ | PSNR (all pixels): 35.67 -> 35.00 | Hurt |
| use SoftShadow instead of BCDiff | zero-shot shadow removal on LRSS · LRSS | PSNR: 22.13 -> 23.32 | Helped |
| use SoftShadow instead of G2R-ShadowNet | zero-shot shadow removal on UIUC · UIUC | PSNR: 27.56 -> 28.85 | Helped |
| add shadow removal loss Lrem (ablation) | SoftShadow · SRD | PSNR: 31.27 -> 35.27 | Helped |
| add mask reconstruction loss Lmask (ablation) | SoftShadow with Lrem · SRD | PSNR: 35.27 -> 35.44 | Helped |
| add penumbra formation constraint loss Lpen (ablation) | SoftShadow with Lrem and Lmask · SRD | PSNR: 35.44 -> 35.57 | Helped |
| add shadow removal loss Lrem (ablation) | SoftShadow · LRSS | PSNR: 21.40 -> 21.78 | Helped |
| add mask reconstruction loss Lmask (ablation) | SoftShadow with Lrem · LRSS | PSNR: 21.78 -> 23.08 | Helped |
| add penumbra formation constraint loss Lpen (ablation) | SoftShadow with Lrem and Lmask · LRSS | SSIM: 0.935 -> 0.933 | Hurt |
Surprising findings
- DeS3 has better results on LRSS because it was trained on the LRSS training set, while SoftShadow was only trained on SRD.
- Prior SOTA methods such as ShadowDiffusion and HomoFormer drop notably in PSNR when given generated masks instead of ground-truth masks, while SoftShadow stays stable across mask types.
Limitations
- Gains on ISTD+ are limited because that dataset has more hard shadows, where soft masks offer less advantage; PSNR is slightly below ground-truth-mask methods.
- Evaluated only at 256x256 resolution following ShadowDiffusion settings.
- Penumbra constraint relies on hand-set thresholds (t1, t2, t) and the ground-truth soft mask is derived from a thresholded Y-channel ratio.
Experimental setup
Datasets: SRD ISTD+ LRSS UIUC
Benchmarks: SRD ISTD+ LRSS UIUC
Metrics: PSNR SSIM MAE
Baselines: DC-ShadowNet DeS3 SAM-helps-shadow DHAN BMNet SG-ShadowNet ShadowFormer ShadowDiffusion Inpaint4Shadow HomoFormer BCDiff G2R-ShadowNet
Builds on: Segment Anything Model LoRA ShadowDiffusion HomoFormer DHAN
Closest papers by meaning
- Polarization Guided Mask-Free Shadow Removal (AAAI 2025 · similarity 0.88)
- DeS3: Adaptive Attention-Driven Self and Soft Shadow Removal Using ViT Similarity (AAAI 2024 · similarity 0.87)
- When Shadow Removal Meets Intrinsic Image Decomposition: A Joint Learning Framework Using Unpaired Data (AAAI 2025 · similarity 0.86)
- MetaShadow: Object-Centered Shadow Detection, Removal, and Synthesis (CVPR 2025 · similarity 0.86)
- Detail-Preserving Latent Diffusion for Stable Shadow Removal (CVPR 2025 · similarity 0.85)
- Recasting Regional Lighting for Shadow Removal (AAAI 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.