Research map / Detection and segmentation / Open-set and few-shot detection
MetaShadow: Object-Centered Shadow Detection, Removal, and Synthesis
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces MetaShadow, a three-in-one object-centered framework combining a GAN-based Shadow Analyzer (detection and removal) with a reference-based diffusion Shadow Synthesizer that receives shadow features from the analyzer.
Problem
Shadow detection, removal, and synthesis are treated separately in prior work, and few methods handle them jointly in an object-centered way for image editing (object removal, relocation, insertion).
Approach
Two cooperating stages: a CM-GAN-based Shadow Analyzer detects and removes an object's shadow given an object mask, and its multi-scale intermediate features are transferred as the condition for a diffusion-based Shadow Synthesizer, replacing text/CLIP embeddings, so shadow knowledge is shared across tasks.
Key results
- Shadow detection mIoU improves from 55.8 (SSISv2) to 71.0 on the SOBA test set.
- Shadow removal Bbox PSNR of 32.97 on DESOBA vs 24.28 for fine-tuned ShadowDiffusion with GT mask.
- Shadow synthesis Local RMSE reduced to 36.84 on DESOBA (256x256) and 36.54 on Video DESOBA vs 51.73 for SGRNet on Video DESOBA.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add MOS Dataset to training (ablation) | Shadow Analyzer · SOBA test set | mIoU: 67.2 -> 71.0 | Helped |
| Use Shadow Analyzer features as diffusion conditioning instead of text embeddings (ablation) | Shadow Synthesizer · DESOBA test set | Global RMSE: 3.36 -> 2.93 | Helped |
| Use Shadow Analyzer features as diffusion conditioning instead of fine-tuned CLIP image embeddings (ablation) | Shadow Synthesizer · DESOBA test set | Global RMSE: 4.51 -> 2.93 | Helped |
| Use 128 × 128 instead of 256 × 256 synthesis resolution (ablation) | MetaShadow · DESOBA test set | Local RMSE: 36.84 -> 30.92 | Helped |
| Use 128 × 128 instead of 256 × 256 synthesis resolution (ablation) | MetaShadow · DESOBA test set | Bbox PSNR: 29.16 -> 30.73 | Helped |
| Use MetaShadow instead of the strongest listed baseline | Object-centered shadow detection · SOBA test set | mIoU: 55.8 -> 71.0 | Helped |
| Use MetaShadow instead of ShadowDiffusion with GT shadow mask | Object-centered shadow removal · DESOBA test set | Bbox PSNR: 24.28 -> 32.97 | Helped |
| Use MetaShadow instead of SGRNet | Object-centered shadow synthesis · DESOBA test set | Local RMSE: 56.44 -> 30.92 | Helped |
| Use MetaShadow instead of SGRNet | Object-centered shadow synthesis · Video DESOBA | Local RMSE: 51.73 -> 36.54 | Helped |
Surprising findings
- Image embeddings (fine-tuned CLIP) and text embeddings from T5 were compared, with the task-specific shadow features outperforming both as conditioning for synthesis.
- SGDiffusion produces inconsistent shadows across random seeds.
Limitations
- Shadow synthesis operates at 128x128 resolution internally, upsampled for comparison.
- Reliance on reference object or shadow for synthesis control.
Experimental setup
Datasets: MOS (new synthetic) DESOBA ISTD+ SRD Shadow-AR SOBA
Benchmarks: SOBA DESOBA Moving DESOBA Video DESOBA ISTD+
Metrics: mIoU Masked MAE Masked RMSE Bbox PSNR Bbox SSIM PSNR Global RMSE Local RMSE
Baselines: SSIS SSISv2 ShadowDiffusion HomoFormer RASM SGRNet SGDiffusion Libcom
Builds on: CM-GAN Stable Diffusion inpainting SGRNet ShadowDiffusion SSISv2
Closest papers by meaning
- Recasting Regional Lighting for Shadow Removal (AAAI 2024 · similarity 0.86)
- Polarization Guided Mask-Free Shadow Removal (AAAI 2025 · similarity 0.86)
- OmniSR: Shadow Removal Under Direct and Indirect Lighting (AAAI 2025 · similarity 0.86)
- When Shadow Removal Meets Intrinsic Image Decomposition: A Joint Learning Framework Using Unpaired Data (AAAI 2025 · similarity 0.86)
- SoftShadow: Leveraging Soft Masks for Penumbra-Aware Shadow Removal (CVPR 2025 · similarity 0.86)
- Shadow-Enlightened Image Outpainting (CVPR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.