Research map / Detection and segmentation / CNN features
DeS3: Adaptive Attention-Driven Self and Soft Shadow Removal Using ViT Similarity
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes DeS3, a mask-free conditional DDIM for single-image removal of hard, soft and self shadows, using classifier-driven adaptive attention and a DINO-ViT self-similarity loss applied during reverse sampling.
Problem
Removing soft and self shadows that lack clear boundaries from a single image, where binary shadow masks and ground-truth shadow-free images are hard to obtain.
Approach
Replaces binary shadow masks with an adaptive, progressively refined classifier-driven CAM attention inside a conditional DDIM, and uses a DINO-ViT key self-similarity loss as a self-tuned stopping criterion during inference to preserve object structures.
Key results
- On SRD, overall RMSE 3.72 vs 4.66 for DC and 5.71 for DSC; PSNR 34.11 and SSIM 0.968 overall.
- On LRSS, RMSE reduced from 3.49 (ShadowDiffusion) to 3.01, a 16% reduction, with PSNR 33.95.
- User study on UIUC self shadows: DeS 8.6 and Real 9.0, highest among compared methods.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use DeS3 instead of ShadowDiffusion | Shadow removal · LRSS | RMSE: 3.49 -> 3.01 | Helped |
| Use DeS3 instead of DC | Shadow removal · LRSS | PSNR: 31.01 -> 33.95 | Helped |
| Use DeS3 instead of ShadowDiffusion | Shadow removal · LRSS | RMSE: from 3.49 to 3.01 | Helped |
| Add adaptive classifier attention (ablation) | DeS3 · SRD shadow region | RMSE: 6.35 -> 5.88 | Helped |
| Add adaptive classifier attention (ablation) | DeS3 · SRD non-shadow region | RMSE: 3.02 -> 2.83 | Helped |
| Add adaptive classifier attention (ablation) | DeS3 · SRD entire image | RMSE: 4.46 -> 3.72 | Helped |
| Add ViT similarity loss (ablation) | DeS3 · SRD shadow region | RMSE: 6.21 -> 5.88 | Helped |
| Add ViT similarity loss (ablation) | DeS3 · SRD non-shadow region | RMSE: 2.99 -> 2.83 | Helped |
| Add ViT similarity loss (ablation) | DeS3 · SRD entire image | RMSE: 3.94 -> 3.72 | Helped |
Limitations
- Adaptive attention and ViT loss rely on heuristic choices (lambda_sim=2, alpha and beta set empirically).
- Self-shadow evaluation is qualitative or user-study based since ground truth for outdoor self shadows is unavailable.
Experimental setup
Datasets: SRD AISTD LRSS USR UIUC UCF
Benchmarks: SRD AISTD LRSS USR UIUC
Metrics: RMSE PSNR SSIM user study score
Baselines: DHAN Auto EM BM SG DC SR3 WeatherDiffusion DSC ShadowDiffusion SP+M Guo
Builds on: DDIM DDPM Class Activation Map DINO-ViT ShadowDiffusion
Closest papers by meaning
- Polarization Guided Mask-Free Shadow Removal (AAAI 2025 · similarity 0.87)
- SoftShadow: Leveraging Soft Masks for Penumbra-Aware Shadow Removal (CVPR 2025 · similarity 0.87)
- Detail-Preserving Latent Diffusion for Stable Shadow Removal (CVPR 2025 · similarity 0.86)
- OmniSR: Shadow Removal Under Direct and Indirect Lighting (AAAI 2025 · similarity 0.86)
- Diff-Shadow: Global-guided Diffusion Model for Shadow Removal (AAAI 2025 · similarity 0.86)
- When Shadow Removal Meets Intrinsic Image Decomposition: A Joint Learning Framework Using Unpaired Data (AAAI 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.