Research map / Image generation / Text-to-image diffusion
VODiff: Controlling Object Visibility Order in Text-to-Image Generation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes VODiff, a training-free layout-guided text-to-image method that controls object occlusion via visibility order, using sequential per-object denoising stages and an attention-based visibility-order-aware loss, and introduces the VOBench benchmark.
Problem
Layout-guided text-to-image diffusion models treat all objects as one layer and ignore visibility order, producing incorrect occlusions among overlapping objects.
Approach
Assigns each user-specified object to a separate layer ordered by visibility, denoises objects sequentially bottom-to-top in sub-stages with smoothly varying visual guidance (SDP), and optimizes cross-attention maps with a loss that separates overlapping, visible, and background regions per object (VOA loss).
Key results
- VODiff achieves OA 82.50 and LA 55.11 on VOBench, versus 69.50 OA for the best training-based competitor (AnyControl).
- Training-free VODiff reaches CLIP-Score 29.73 and FID 10.03, comparable to MIGC's best FID of 9.82.
- Ablation shows removing the VOA loss or SDP lowers occlusion accuracy (e.g., SDP+VOA gives OA 82.50 vs 73.50 with SDP only).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add SDP to the method without VOA (first Table 2 variant) (ablation) | VODiff · VOBench | FID: 9.98 -> 10.03 | No clear effect |
| Add VOA to SDP with dynamic guidance (ablation) | VODiff · VOBench | OA: 73.50 -> 82.50 | Helped |
| Replace constant SDP guidance with dynamic guidance (ablation) | VODiff with VOA · VOBench | OA: 69.50 -> 82.50 | Helped |
| Add the occluded-region loss to visible-region loss (ablation) | VODiff · VOBench | OA: 60.50 -> 61.50 | Helped |
| Add the background-region loss to visible-region loss (ablation) | VODiff · VOBench | OA: 60.50 -> 77.50 | Helped |
| Add the visible-region loss to occluded- and background-region losses (ablation) | VODiff · VOBench | LA: 29.14 -> 55.11 | Helped |
| Add the background-region loss to visible- and occluded-region losses (ablation) | VODiff · VOBench | FID: 10.23 -> 10.03 | Helped |
| Add the occluded-region loss to visible- and background-region losses (ablation) | VODiff · VOBench | OA: 77.50 -> 82.50 | Helped |
| Add the background-region loss to visible- and occluded-region losses (ablation) | VODiff · VOBench | LA: 46.42 -> 55.11 | Helped |
Surprising findings
- Using only the SDP component gives the worst layout and occlusion accuracy despite better FID, since it only nudges denoising direction without strong constraints.
Limitations
- Struggles when the number of objects is very high, balancing quality with precise spatial and occlusion constraints.
- Occlusion evaluation relies on GPT-4V judgments and the benchmark is small (200 samples).
Experimental setup
Datasets: VOBench
Benchmarks: VOBench
Metrics: FID CLIP-Score Layout Alignment (mIoU via Grounding-DINO) Occlusion Alignment (GPT-4V) user study average ranking
Baselines: BoxDiff R&B Attention-Refocusing MultiDiffusion DenseDiffusion FreeControl RPG-Diffusion SmartControl ControlNet AnyControl MIGC InstanceDiffusion
Builds on: Stable Diffusion MultiDiffusion Attention Refocusing Layout guidance (BoxDiff/R&B)
Closest papers by meaning
- HDLayout: Hierarchical and Directional Layout Planning for Arbitrary Shaped Visual Text Generation (AAAI 2025 · similarity 0.83)
- Zero-Painter: Training-Free Layout Control for Text-to-Image Synthesis (CVPR 2024 · similarity 0.83)
- PreciseCam: Precise Camera Control for Text-to-Image Generation (CVPR 2025 · similarity 0.82)
- LLM Blueprint: Enabling Text-to-Image Generation with Complex and Detailed Prompts (ICLR 2024 · similarity 0.82)
- Reproducibility Study of "ITI-GEN: Inclusive Text-to-Image Generation" (NeurIPS 2024 · similarity 0.82)
- PosterMaker: Towards High-Quality Product Poster Generation with Accurate Text Rendering (CVPR 2025 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.