Research map / Image generation / Editing and style transfer
Generative Image Layer Decomposition with Visual Effects
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes LayerDecomp, a diffusion-transformer framework that decomposes an image with an object mask into a clean background and a transparent RGBA foreground that keeps shadows and reflections, trained on simulated triplets plus camera-captured pairs with a pixel-space consistency loss.
Problem
Decomposing a single image into editable layers (clean background and transparent foreground) while faithfully preserving transparent visual effects such as shadows and reflections, without ground-truth layer annotations for real images.
Approach
A DiT generates background and RGBA foreground latents jointly from the composite image and object mask; for real data lacking foreground ground truth, a pixel-space consistency loss requires the alpha-blended re-composite of the predicted layers to match the input image.
Key results
- On RORD, LayerDecomp reaches 24.56 PSNR (loose mask) vs. 20.88 for the best mask-based baseline (PowerPaint), with FID 21.77 vs 39.48.
- In a 17-researcher user study on object removal, LayerDecomp is preferred overall in 83.54% (vs Emu-Edit) and 87.78% (vs OmniGen) of cases.
- In the ablation, adding the consistency loss raises composite-image PSNR from 28.66 to 30.53 over the RGBA-foreground variant without it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add an RGBA foreground layer containing the object (ablation) | LayerDecomp · Held-out test set, BG | PSNR: 28.21 -> 28.28 | Helped |
| add visual effects to the RGBA foreground layer (ablation) | LayerDecomp · Held-out test set, Comp | PSNR: 27.53 -> 28.66 | Helped |
| add consistency loss (ablation) | LayerDecomp · Held-out test set, Comp | PSNR: 28.66 -> 30.53 | Helped |
| use LayerDecomp instead of mask-based inpainting | object removal · RORD | PSNR: 24.56L24.79T vs. 20.45L22.01T | Helped |
| use LayerDecomp instead of mask-based inpainting | object removal · MULAN | FID: 39.26 vs. 55.80 | Helped |
| use LayerDecomp instead of mask-based inpainting | shadow removal · DESOBAv2 | PSNRm: 38.57T vs. 36.94L | Helped |
| use LayerDecomp for object removal instead of Emu-Edit | instruction-driven object removal · Emu-Edit Remove Set | overall preference: 5.00% vs. 57.08% (removal effectiveness); 4.79% vs. 83.54% (overall) | Helped |
| use LayerDecomp for object moving instead of DesignEdit | object spatial editing · Moving user study | overall preference: 2.22% vs. 94.44% | Helped |
Limitations
- Struggles with incomplete masks, heavy occlusions, and intricate colorful shadows.
- Simulated composites may lack realistic geometry and harmonized content.
Experimental setup
Datasets: simulated multi-layer triplets (100k foreground assets, 5M backgrounds) camera-captured image pairs (6,000) RORD MULAN DESOBAv2 Emu-Edit Remove Set
Benchmarks: RORD MULAN DESOBAv2 Emu-Edit Remove Set
Metrics: PSNR LPIPS FID CLIP-FID masked PSNR masked SSIM user preference
Baselines: ControlNet Inpainting SD-XL Inpainting PowerPaint ObjectDrop Emu-Edit MGIE OmniGen DiffEditor DragAnything DesignEdit DiffusionHandle
Builds on: diffusion transformer LayerDiffusion ObjectDrop MULAN
Closest papers by meaning
- Generative Omnimatte: Learning to Decompose Video into Layers (CVPR 2025 · similarity 0.88)
- Move Anything with Layered Scene Diffusion (CVPR 2024 · similarity 0.86)
- Decomposition of Graphic Design with Unified Multimodal Model (ICML 2025 · similarity 0.86)
- LayerCraft: Enhancing Text-to-Image Generation with CoT Reasoning and Layered Object Integration (NeurIPS 2025 · similarity 0.85)
- ART: Anonymous Region Transformer for Variable Multi-Layer Transparent Image Generation (CVPR 2025 · similarity 0.84)
- PixelART: Image-to-Layer Decomposition without Latents or Text-to-Image Pretraining (NeurIPS 2026 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.