Research map / Image generation / Text-to-image diffusion
ART: Anonymous Region Transformer for Variable Multi-Layer Transparent Image Generation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes ART, a FLUX.1-dev-based diffusion transformer that generates variable multi-layer transparent images from a global prompt and an anonymous (unlabeled) region layout, with layer-wise regional cropped attention and a multi-layer transparency autoencoder.
Problem
Generating multiple transparent image layers at variable counts and resolutions with coherent harmonization across layers, while avoiding the labor of semantic layouts with per-region prompts and the cost of full attention over many layers.
Approach
Replace semantic region-wise prompts with an anonymous region layout (only bounding boxes), let the transformer infer each region's semantics from the global prompt (schema-theory motivated), crop each layer's tokens to its box for regional full attention, and decode all RGBA layers jointly with a ViT using 3D RoPE over (x, y, layer) indices.
Key results
- Regional full attention reaches FIDmerged 17.79 and PSNR 22.90 vs 41.35 / 16.87 for full attention, and keeps near-constant cost from 10 to 50 layers.
- In user studies (30 participants), ART beats LayerDiffuse on photo multi-layer images and COLE on design images across aesthetics, harmonization, and prompt following.
- The multi-layer transparency decoder reaches PSNR 30.48 and FIDmerged 2.84, versus 26.51 and 4.27 for LayerDiffuse's decoder.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace semantic region prompts with anonymous regions (ablation) | ART · DESIGN-MULTI-LAYER-BENCH validation | PSNR: 17.71 -> 22.90 | Helped |
| replace semantic region prompts with anonymous regions (ablation) | ART · DESIGN-MULTI-LAYER-BENCH validation | FIDmerged: 17.51 -> 17.79 | No clear effect |
| add composed image prediction (ablation) | ART · DESIGN-MULTI-LAYER-BENCH validation | FIDmerged: 20.44 -> 17.79 | Helped |
| add composed image prediction (ablation) | ART · DESIGN-MULTI-LAYER-BENCH validation | Inference speed (s): 19.20 -> 26.62 | Hurt |
| use regional full attention instead of full attention without cropping (ablation) | ART · DESIGN-MULTI-LAYER-BENCH validation | FIDmerged: 41.35 -> 17.79 | Helped |
| use regional full attention instead of spatial plus temporal attention (ablation) | ART · DESIGN-MULTI-LAYER-BENCH validation | FIDmerged: 167.99 -> 17.79 | Helped |
| add layer-aware position information using 3D-RoPE instead of 2D-RoPE (ablation) | ART · DESIGN-MULTI-LAYER-BENCH validation | FIDmerged: 124.3 -> 17.79 | Helped |
| increase training data from 80 to 800k samples (ablation) | ART · DESIGN-MULTI-LAYER-BENCH validation | FIDmerged: 30.38 -> 17.79 | Helped |
| use anonymous rather than semantic layout planner (ablation) | ART · DESIGN-MULTI-LAYER-BENCH validation | FIDmerged: 21.45 -> 19.90 | Helped |
| remove predicted region-specific prompts from the semantic layout planner (ablation) | ART · DESIGN-MULTI-LAYER-BENCH validation | PSNR: 17.69 -> 22.90 | Helped |
| replace absolute position encoding with 3D-RoPE in the multi-layer decoder (ablation) | ART multi-layer decoder · DESIGN-MULTI-LAYER-BENCH validation | PSNRlayer RGB: 26.91 -> 34.89 | Helped |
| condition the multi-layer decoder on both composed and background images instead of neither (ablation) | ART multi-layer decoder · DESIGN-MULTI-LAYER-BENCH validation | PSNRlayer alpha: 22.82 -> 23.85 | Helped |
Surprising findings
- Over 80% of labels inferred for anonymous regions agree with human-annotated region prompts.
- Adding region-specific prompts for each layer did not improve results and reduced cross-layer coherence.
Limitations
- Repeated layer generation and combined layer generation remain failure modes.
- Generalization across all possible anonymous layouts is not yet established.
- Training data (MLTD) is private and the semantic labels of regions are not learned explicitly.
Experimental setup
Datasets: MLTD (private multi-layer transparent design dataset, ~1M) COCO
Benchmarks: DESIGN-MULTI-LAYER-BENCH PHOTO-MULTI-LAYER-BENCH
Metrics: FID PSNR SSIM user study win rate harmonization score inference speed GPU memory
Baselines: LayerDiffuse COLE LayoutGPT Text2Layer Flux-RGBA decoder GPT-4o
Builds on: FLUX.1-dev LayerDiffuse RoPE Prodigy
Closest papers by meaning
- Scene Graph Disentanglement and Composition for Generalizable Complex Image Generation (NeurIPS 2024 · similarity 0.85)
- LayerCraft: Enhancing Text-to-Image Generation with CoT Reasoning and Layered Object Integration (NeurIPS 2025 · similarity 0.85)
- Rethinking The Training And Evaluation of Rich-Context Layout-to-Image Generation (NeurIPS 2024 · similarity 0.85)
- Generating compositional scenes via Text-to-image RGBA Instance Generation (NeurIPS 2024 · similarity 0.85)
- Zero-Painter: Training-Free Layout Control for Text-to-Image Synthesis (CVPR 2024 · similarity 0.84)
- Generative Image Layer Decomposition with Visual Effects (CVPR 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.