Research map / Image generation / Autoregressive and latent generation
Shadow Generation Using Diffusion Model with Geometry Prior
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes GPSDiffusion, a ControlNet-based shadow generation model that injects predicted rotated shadow bounding boxes and retrieved prototypical shadow shape embeddings as geometry priors to improve shadow placement and shape.
Problem
Diffusion-based shadow generation for inserted foreground objects produces realistic shadow appearance but poor shadow geometry (location, scale, shape) in complex cases.
Approach
A multi-task geometry encoder predicts a rotated bounding box regression for the foreground shadow and retrieves top-K matched shadow shapes via clustered autoencoder prototypes; the box is appended as a binary mask to the ControlNet encoder input and shape embeddings are injected through added cross-attention layers in the U-Net.
Key results
- On DESOBAv2 BOS-free test images, GPSDiffusion reaches GR 13.809 vs 15.110 for SGDiffusion and LR 55.616 vs 55.874 (LR is marginally worse than the 53.611 BOS value for SGDiffusion in BOS images; best overall on most metrics).
- Rotated bounding box regression KFIoU improves from 0.203 (DMASNet) to 0.225.
- Ablation on BOS test images: full model GR 5.896 and LR 46.713 vs base ControlNet GR 7.567 and LR 54.610.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add the rotated bounding-box prior (ablation) | GPSDiffusion · DESOBAv2 BOS Test Images | Global RMSE (GR): 7.567 -> 6.331 | Helped |
| replace the rotated bounding box with a non-rotated bounding box (ablation) | GPSDiffusion · DESOBAv2 BOS Test Images | Global RMSE (GR): 6.331 -> 7.100 | Hurt |
| add matched shadow-shape embeddings (ablation) | GPSDiffusion · DESOBAv2 BOS Test Images | Global RMSE (GR): 7.567 -> 6.363 | Helped |
| add bounding-box and shape priors (ablation) | ControlNet base · DESOBAv2 BOS Test Images | Global RMSE (GR): 7.567 -> 5.896 | Helped |
| use GPSDiffusion instead of SGDiffusion | DESOBAv2 BOS Test Images · DESOBAv2 BOS Test Images | Global RMSE (GR): 6.098 -> 5.896 | Helped |
| use GPSDiffusion instead of SGDiffusion | DESOBAv2 BOS Test Images · DESOBAv2 BOS Test Images | Global SSIM (GS): 0.971 -> 0.966 | Hurt |
| use GPSDiffusion instead of SGDiffusion | DESOBAv2 BOS-free Test Images · DESOBAv2 BOS-free Test Images | Local SSIM (LS): 0.117 -> 0.166 | Helped |
Surprising findings
- Non-rotated bounding box gives smaller gains than rotated bounding box in ablation, while shape embeddings alone also help.
Limitations
- Evaluation limited to DESOBAv2 and a small set of real composite images
- Shadow shapes restricted to a discrete set of prototypes (N=256 clusters) via retrieval
Experimental setup
Datasets: DESOBAv2 DESOBA
Benchmarks: DESOBAv2 real composite images from SGRNet
Metrics: Global RMSE Local RMSE Global SSIM Local SSIM Global BER Local BER top-K accuracy KFIoU B-T score
Baselines: ShadowGAN Mask-SG AR-SG SGRNet DMASNet SGDiffusion
Builds on: ControlNet SGDiffusion DMASNet KFIoU loss
Closest papers by meaning
- Shadow Generation for Composite Image Using Diffusion Model (CVPR 2024 · similarity 0.90)
- Shadow Generation with Decomposed Mask Prediction and Attentive Shadow Filling (AAAI 2024 · similarity 0.88)
- Detail-Preserving Latent Diffusion for Stable Shadow Removal (CVPR 2025 · similarity 0.85)
- MirrorVerse: Pushing Diffusion Models to Realistically Reflect the World (CVPR 2025 · similarity 0.84)
- Gaussian Shadow Casting for Neural Characters (CVPR 2024 · similarity 0.84)
- Neural Point Cloud Diffusion for Disentangled 3D Shape and Appearance Generation (CVPR 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.