Research map / 3D vision / 3D generation
OmniSR: Shadow Removal Under Direct and Indirect Lighting
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a path-tracing pipeline that renders shadowed/shadow-free pairs under direct and indirect illumination, releases the 30k-pair INS synthetic dataset, and a semantic- and geometry-attention U-Net (CSA layers) for shadow removal that outperforms prior SOTA.
Problem
Single-image shadow removal methods are trained on data dominated by direct illumination shadows and fail on subtle indirect shadows in indoor scenes, partly because no shadow-free ground truth exists for indirect shadows.
Approach
Render shadow and shadow-free pairs for both direct and indirect illumination with a modified path tracer (removing first-bounce occluders within 1 m for indirect shadow-free images), and use a Swin-style U-Net whose attention is modulated by DINO-feature semantic similarity and depth-derived planar geometric distance, with DINO features concatenated at the bottleneck.
Key results
- On ISTD, the method reaches 30.45 PSNR / 0.964 SSIM without ground-truth masks, and 33.34 PSNR on ISTD+, the best among compared methods.
- On the INS test set the method reaches 30.38 PSNR / 0.973 SSIM, above ShadowDiffusion's 29.12 / 0.966.
- Ablation: removing DINO concatenation drops WRSD+ PSNR from 26.07 to 23.31, and removing depth drops INS testing PSNR from 30.38 to 29.95.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add depth input (ablation) | OmniSR · INS testing | PSNR: 29.95 -> 30.38 | Helped |
| Add depth input (ablation) | OmniSR · Real captured | PSNR: 38.03 -> 38.34 | Helped |
| Add depth input (ablation) | OmniSR · WRSD+ | PSNR: 25.73 -> 26.07 | Helped |
| Add DINO feature concatenation (ablation) | OmniSR · INS testing | PSNR: 29.31 -> 30.38 | Helped |
| Add DINO feature concatenation (ablation) | OmniSR · Real captured | PSNR: 37.06 -> 38.34 | Helped |
| Add DINO feature concatenation (ablation) | OmniSR · WRSD+ | PSNR: 23.31 -> 26.07 | Helped |
| Add semantic attention weights (ablation) | OmniSR · INS testing | PSNR: 30.01 -> 30.38 | Helped |
| Add semantic attention weights (ablation) | OmniSR · Real captured | PSNR: 38.18 -> 38.34 | Helped |
| Add semantic attention weights (ablation) | OmniSR · WRSD+ | PSNR: 25.96 -> 26.07 | Helped |
| Add geometric attention weights (ablation) | OmniSR · INS testing | PSNR: 30.22 -> 30.38 | Helped |
| Add geometric attention weights (ablation) | OmniSR · Real captured | PSNR: 37.82 -> 38.34 | Helped |
| Add geometric attention weights (ablation) | OmniSR · WRSD+ | PSNR: 25.77 -> 26.07 | Helped |
Surprising findings
- Adding the synthetic INS data did not fully fix diffusion or transformer baselines: DMTN, ShadowFormer and ShadowDiffusion still leave complex indoor shadows, per the authors.
- The ablation without the object-composition scenes lowers INS test PSNR from 30.38 to 29.32, showing the simpler synthetic scenes help despite lower realism.
Limitations
- May fail to remove very large shadow regions, such as those covering entire surfaces or objects.
- Higher computational cost from depth and DINO feature extractors compared with lightweight methods.
Experimental setup
Datasets: INS (synthetic, rendered) ISTD ISTD+ SRD WRSD+ 3DFront ABO Objaverse ambientCG
Benchmarks: ISTD ISTD+ SRD WRSD+ INS real captured indoor images
Metrics: PSNR SSIM
Baselines: DSC DHAN Fu et al. 2021 Zhu et al. 2022 (BMNet/bijective) BMNet ShadowFormer DMTN ShadowDiffusion ShadowRefiner
Builds on: ShadowFormer Swin Transformer DINO V2 Depth-Anything-V2 Blender Cycles
Closest papers by meaning
- Recasting Regional Lighting for Shadow Removal (AAAI 2024 · similarity 0.89)
- When Shadow Removal Meets Intrinsic Image Decomposition: A Joint Learning Framework Using Unpaired Data (AAAI 2025 · similarity 0.88)
- Polarization Guided Mask-Free Shadow Removal (AAAI 2025 · similarity 0.86)
- DeS3: Adaptive Attention-Driven Self and Soft Shadow Removal Using ViT Similarity (AAAI 2024 · similarity 0.86)
- Detail-Preserving Latent Diffusion for Stable Shadow Removal (CVPR 2025 · similarity 0.86)
- MetaShadow: Object-Centered Shadow Detection, Removal, and Synthesis (CVPR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.