Research map / Image generation / Text-to-image diffusion
HDLayout: Hierarchical and Directional Layout Planning for Arbitrary Shaped Visual Text Generation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes HDLayout, a hierarchical (region, line, character) and directional layout representation with Bézier curves, plugged into a separation-and-composition pipeline that generates arbitrarily shaped visual text from text prompts alone, with a new HDLayout3k dataset.
Problem
Visual text generation in scene images is limited by rectangular bounding-box layouts and the need for manual layout inputs, which cannot capture arbitrarily shaped, multi-granularity text structures.
Approach
Represent visual text hierarchically (region boxes, line boxes, character-level Bézier curves) with a coarse-to-fine transformer decoder conditioned on the generated background image, then render characters onto the background and refine with a diffusion model.
Key results
- Full HDLayout achieves Image FID 78.027 vs. 82.068 for TextDiffuser and 92.462 for SD-2.1 on Anytext-benchmark.
- Layout FID of 12.343 for HDLayout vs. 27.135 for TextDiffuser.
- Removing both region and line levels raises Image FID to 115.84, showing hierarchical decoding matters.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use HDLayout instead of TextDiffuser | Visual text generation with SD-1.5 · Anytext-benchmark | Image FID: 82.068 -> 78.027 | Helped |
| use HDLayout instead of TextDiffuser | Visual text generation with SD-1.5 · Anytext-benchmark | Layout FID: 27.135 -> 12.343 | Helped |
| add region and line decoders (ablation) | HDLayout · Anytext-benchmark | Image FID: 115.84 -> 78.027 | Helped |
| add region and line decoders (ablation) | HDLayout · Anytext-benchmark | Layout FID: 16.474 -> 12.343 | Helped |
| add region decoder (ablation) | HDLayout · Anytext-benchmark | Image FID: 102.67 -> 78.027 | Helped |
| add region decoder (ablation) | HDLayout · Anytext-benchmark | Layout FID: 16.094 -> 12.343 | Helped |
| add line decoder (ablation) | HDLayout · Anytext-benchmark | Image FID: 113.13 -> 78.027 | Helped |
| add line decoder (ablation) | HDLayout · Anytext-benchmark | Layout FID: 15.697 -> 12.343 | Helped |
| use the full HDLayout rendering pipeline instead of ControlNet with HDLayout layout guidance (ablation) | HDLayout · Anytext-benchmark | Image FID: 88.920 -> 78.027 | Helped |
| use the full HDLayout rendering pipeline instead of GlyphControl with HDLayout layout guidance (ablation) | HDLayout · Anytext-benchmark | Image FID: 86.842 -> 78.027 | Helped |
Limitations
- Performance degrades for small text areas due to difficulty determining accurate character positions.
- Bézier curve loss is a simple L1 loss with no dedicated design for curves.
Experimental setup
Datasets: HDLayout3k SCUT-EnxText Anytext-benchmark
Benchmarks: Anytext-benchmark
Metrics: FID Layout FID
Baselines: SD-2.1 SD-XL ControlNet GlyphControl TextDiffuser
Builds on: DETR ControlNet TextDiffuser GlyphControl
Closest papers by meaning
- Rethinking The Training And Evaluation of Rich-Context Layout-to-Image Generation (NeurIPS 2024 · similarity 0.88)
- Layout-Agnostic Scene Text Image Synthesis with Diffusion Models (CVPR 2024 · similarity 0.87)
- Layout-your-3D: Controllable and Precise 3D Generation with 2D Blueprint (ICLR 2025 · similarity 0.87)
- Build-A-Scene: Interactive 3D Layout Control for Diffusion-Based Image Generation (ICLR 2025 · similarity 0.86)
- Seeing is Improving: Visual Feedback for Iterative Text Layout Refinement (CVPR 2026 · similarity 0.86)
- LLM Blueprint: Enabling Text-to-Image Generation with Complex and Detailed Prompts (ICLR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.