Research map / Image generation / Autoregressive and latent generation
Towards Aligned Layout Generation via Diffusion Model with Aesthetic Constraints
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes LACE, a continuous-space diffusion model for unified controllable layout generation that adds differentiable global alignment and overlap constraints to training and post-processing, using masked inputs for conditioning.
Problem
Diffusion-based layout generation achieves strong FID but shows poor alignment and overlap compared with transformer models, and discrete diffusion cannot exploit differentiable aesthetic constraints on quantized geometry.
Approach
Model labels and bounding boxes jointly in continuous space with a DDPM, so differentiable alignment and overlap losses can be applied to the predicted x0; conditioning is handled by masking known attributes in the noisy input, and a time-dependent weight enables constraints only at low-noise steps.
Key results
- On PubLayNet C->S+P, LACE (local) reaches FID 4.88 vs LayoutDM 7.95; LACE (global) w/ post gives alignment 0.007-level improvement over LayoutDM w/ post (0.009 vs 0.083 alignment in C->S+P table).
- On PubLayNet unconditional generation, LACE (local) w/ post reduces alignment to 0.032 from 0.195 for LayoutDM, while FID stays at 8.47.
- On refinement (Rico), LACE (local) achieves FID 1.79 and MaxIoU 0.485 vs RUITE 263.23 and 0.421.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add aesthetic constraints to LACE (ablation) | LACE · PubLayNet unconditional | Alignment: 0.238 -> 0.141 | Helped |
| Add aesthetic constraints to LACE (ablation) | LACE · PubLayNet unconditional | FID: 8.70 -> 8.45 | Helped |
| Add aesthetic constraints to LACE (ablation) | LACE · PubLayNet C→S+P | FID: 5.08 -> 4.88 | Helped |
| Add aesthetic constraints to LACE (ablation) | LACE · PubLayNet C+S→P | FID: 3.21 -> 2.80 | Helped |
| Add aesthetic constraints to LACE (ablation) | LACE · PubLayNet C+S→P | MaxIoU: 0.460 -> 0.437 | Hurt |
| Add post-processing (ablation) | LACE with constraints · PubLayNet unconditional | Alignment: 0.141 -> 0.032 | Helped |
| Add post-processing (ablation) | LACE without constraints · PubLayNet unconditional | Alignment: 0.238 -> 0.215 | Helped |
| Add post-processing (ablation) | LACE with constraints · PubLayNet unconditional | FID: 8.45 -> 8.47 | No clear effect |
| Use global rather than local alignment constraint | LACE · PubLayNet C→S+P | MaxIoU: 0.331 -> 0.383 | Helped |
| Use global rather than local alignment constraint | LACE · Rico C→S+P | FID: 3.31 -> 3.24 | Helped |
| Use global rather than local alignment constraint | LACE · Rico C→S+P | MaxIoU: 0.336 -> 0.340 | Helped |
| Use global rather than local alignment constraint | LACE · PubLayNet C→S+P | FID: 4.88 -> 5.14 | Mixed |
Surprising findings
- Applying post-processing to LayoutDM increases its FID notably (e.g., C->S+P 7.95 to 15.2), whereas LACE keeps FID stable after post-processing.
- Directly applying alignment and overlap losses to noisy layouts causes convergence to local minima; without time-dependent weighting FID stays above 100.
Limitations
- Restricts layout elements to rectangular boxes and lacks background/content awareness.
- Handles only a limited number of elements (max 25) and a closed label set.
- Post-processing threshold is set empirically (1/64 of scaled canvas).
Experimental setup
Datasets: PubLayNet Rico
Benchmarks: PubLayNet Rico
Metrics: FID Max IoU Alignment Overlap
Baselines: LayoutVAE NDN-none LayoutGAN++ LayoutTransformer BLT BART MaskGIT VQDiffusion LayoutDM RUITE
Builds on: DDPM LayoutDM LayoutGAN++ DiffMAE RUITE
Closest papers by meaning
- Visual Layout Composer: Image-Vector Dual Diffusion Model for Design Layout Generation (CVPR 2024 · similarity 0.89)
- AesthetiQ: Enhancing Graphic Layout Design via Aesthetic-Aware Preference Alignment of Multi-modal Large Language Models (CVPR 2025 · similarity 0.88)
- Graphic Design with Large Multimodal Model (AAAI 2025 · similarity 0.87)
- DesignDiffusion: High-Quality Text-to-Design Image Generation with Diffusion Models (CVPR 2025 · similarity 0.87)
- Desigen: A Pipeline for Controllable Design Template Generation (CVPR 2024 · similarity 0.87)
- Seeing is Improving: Visual Feedback for Iterative Text Layout Refinement (CVPR 2026 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.