Research map / Image generation / Text-to-image diffusion
Zero-Shot Spatial Layout Conditioning for Text-to-Image Diffusion Models
Guillaume Couairon, Marlène Careil, Matthieu Cord, Stéphane Lathuilière, Jakob Verbeek
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add ZestGuide attention-based guidance over PwW | LDM · COCO-Stuff Eval-few | mIoU: 36.3 -> 46.9 | Improved |
| use ZestGuide instead of external-classifier guidance | LDM · COCO-Stuff Eval-few | mIoU / time: mIoU 20.5 -> 46.9; 1 min -> around 15 secs per image | Improved |
| train-based methods evaluated with only a few segments | OASIS, SDM · COCO-Stuff Eval-few | mIoU: OASIS 52.1 (Eval-all) -> 41.4 (Eval-few) | Hurt |
| average attention across all heads before applying the loss (ablation) | ZestGuide · COCO-Stuff Eval-few | mIoU: 32.1 -> 43.3 | Improved |
| average attention across layers instead of per-layer loss (ablation) | ZestGuide · COCO-Stuff Eval-few | mIoU: 42.7 -> 43.3 | Improved |
| add L-infinity gradient normalization (ablation) | ZestGuide · COCO-Stuff Eval-few | mIoU: 38.99 -> 43.34 | Improved |
| apply guidance for all steps (tau=1) instead of first half (ablation) | ZestGuide · COCO-Stuff Eval-few | mIoU / FID: mIoU 43.34 -> 44.58, FID 31.53 -> 34.75 | Mixed |
| reduce guidance to first 10% of steps (tau=0.1) (ablation) | ZestGuide · COCO-Stuff Eval-few | mIoU: 43.34 -> 34.25 | Hurt |
| restrict attention layers to res16 and res32 (ablation) | ZestGuide · COCO-Stuff Eval-few | mIoU: 40.17 -> 43.34 | Improved |
| combine the normalized-map loss with PwW | ZestGuide · COCO-Stuff Eval-filtered | FID-mIoU trade-off | Improved |
| use class-text embeddings as keys instead of full prompt attention (ablation) | ZestGuide · COCO-Stuff Eval-few | mIoU / FID: mIoU 43.36 vs 43.34, FID about +3 | Hurt |
Earlier methods this paper compares with
Named baselines from earlier accepted papers in this paper's results tables: better than 0 of 1 on most shared results.
- OASIS — OASIS: Only Adversarial Supervision for Semantic Image Synthesis (IJCV 2022) · better on 1 of 3 shared results
Matched by method name in results tables read by AI. Authors' own tables tend to favour the newer method, and a comparison can be missed when a method is renamed.
Closest papers by meaning
- R&B: Region and Boundary Aware Zero-shot Grounded Text-to-image Generation (ICLR 2024 · similarity 0.91)
- SpaText: Spatio-Textual Representation for Controllable Image Generation (CVPR 2023 · similarity 0.90)
- Training-free Composite Scene Generation for Layout-to-Image Synthesis (ECCV 2024 · similarity 0.90)
- Diffusion Self-Distillation for Zero-Shot Customized Image Generation (CVPR 2025 · similarity 0.88)
- Layout-Agnostic Scene Text Image Synthesis with Diffusion Models (CVPR 2024 · similarity 0.88)
- RealCompo: Balancing Realism and Compositionality Improves Text-to-Image Diffusion Models (NeurIPS 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
