Research map / Image generation / Text-to-image diffusion
TextCenGen: Attention-Guided Text-Centric Background Adaptation for Text-to-Image Generation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes TextCenGen, a training-free method that relocates conflicting objects via force-directed cross-attention guidance and applies spatial-excluding attention constraints to generate text-friendly backgrounds in blank regions of T2I images.
Problem
Text-to-image models do not naturally produce backgrounds with clear, smooth blank regions for text placement; directly reducing attention in the target region degrades prompt fidelity.
Approach
Before reducing attention in the text region, detect object tokens whose cross-attention overlaps the region, move them away with force-directed graph repulsion and margin/warping forces on attention maps, then apply a spatial excluding cross-attention constraint to keep the blank region smooth.
Key results
- On the P2P Template dataset, TextCenGen (SD 1.5) reduces saliency IOU to 22.86 versus 29.89 for SD 1.5 (23.52% improvement) and raises VTCM to 4.40 versus 2.95.
- Against Desigen-Trained with attention reduction, TextCenGen achieves lower saliency IOU and TV loss on all three datasets in the trained-comparison table.
- Ablation on P2P: removing both FDG and SEC yields saliency IOU 30.32 and VTCM 2.93, while the full method reaches 22.86 and 4.4.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Apply TextCenGen instead of vanilla Stable Diffusion 1.5 | Stable Diffusion 1.5 · P2P Template, DiffusionDB, and Syn Prompt | VTCM: 2.95 -> 4.4; 3.19 -> 4.39; 2.73 -> 3.49 | Helped |
| Apply TextCenGen instead of vanilla Stable Diffusion 1.5 | Stable Diffusion 1.5 · P2P Template, DiffusionDB, and Syn Prompt | Saliency IOU: 29.89 -> 22.86; 30.11 -> 23.59; 31.42 -> 27.70 | Helped |
| Apply TextCenGen instead of vanilla Stable Diffusion 1.5 | Stable Diffusion 1.5 · P2P Template, DiffusionDB, and Syn Prompt | TV Loss: 14.11 -> 8.81; 12.30 -> 8.41; 15.61 -> 11.37 | Helped |
| Add Force-Directed Cross-Attention Guidance (FDG) (ablation) | TextCenGen without FDG · P2P Template | VTCM: 3.03 -> 4.4 | Helped |
| Add Spatial Excluding Cross-Attention Constraint (SEC) (ablation) | TextCenGen without SEC · P2P Template | TV Loss: 12.76 -> 8.81 | Helped |
Surprising findings
- Directly reducing attention in the target region degrades prompt semantics, motivating relocation of conflicting objects first.
- The trained Desigen model underperforms the training-free TextCenGen on general prompt datasets despite training on graphic design data.
Limitations
- Force-directed guidance assumes convex objects centered on centroids, so non-convex shapes may shrink or fragment.
- Can cause unexpected changes to the subject and unintended objects appearing in the freed space.
Experimental setup
Datasets: Prompt2Prompt templates DiffusionDB ChatGPT-synthesized prompts Desigen validation set
Benchmarks: P2P Template DiffusionDB Syn Prompt Desigen benchmark
Metrics: Saliency IOU TV Loss CLIP Score VTCM CLIP Score Loss ELO design appeal
Baselines: Stable Diffusion 1.5 DALL-E 3 AnyText Desigen Text2Poster
Builds on: Prompt-to-Prompt Desigen stable diffusion
Closest papers by meaning
- Discriminative Probing and Tuning for Text-to-Image Generation (CVPR 2024 · similarity 0.88)
- R&B: Region and Boundary Aware Zero-shot Grounded Text-to-image Generation (ICLR 2024 · similarity 0.88)
- OneIG-Bench: Omni-dimensional Nuanced Evaluation for Image Generation (NeurIPS 2025 · similarity 0.88)
- Style Aligned Image Generation via Shared Attention (CVPR 2024 · similarity 0.88)
- Layout-Agnostic Scene Text Image Synthesis with Diffusion Models (CVPR 2024 · similarity 0.88)
- LLM Blueprint: Enabling Text-to-Image Generation with Complex and Detailed Prompts (ICLR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.