Research map / Image generation / Text-to-image diffusion
Enhancing Semantic Fidelity in Text-to-Image Synthesis: Attention Regulation in Diffusion Models
Yang Zhang, Tze Tzun Teoh, Wei Hern Lim, Kenji Kawaguchi
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| apply inference-time attention regulation to cross-attention maps | Stable Diffusion 2 · COCO subset | Detection rate: 55.9% -> 72.5% | Improved |
| apply attention regulation across four Stable Diffusion versions | SD1.4, SD1.5, SD2, SD2.1 · COCO and A&E datasets | CLIP score: SD1.4 0.323 -> 0.331 | Improved |
| attention regulation versus Attend-And-Excite on image similarity to original | Stable Diffusion 2 · COCO subset | LPIPS: 0.393 -> 0.508 | Hurt |
| attention regulation instead of Attend-And-Excite on compute | Stable Diffusion 2 · COCO subset | computational overhead: 414.6% -> 48.8% | Improved |
| attention regulation on FID relative to original model | Stable Diffusion 2 · FID dataset | FID: 42.72 -> 41.79 | Improved |
| edit more cross-attention layers (ablation) | attention regulation · CLIP similarity ablation | CLIP score | Improved |
| increase number of edited diffusion steps beyond 25 (ablation) | attention regulation · step ablation | CLIP score | No clear effect |
| increase beta regularization up to 0.1 (ablation) | attention regulation · beta ablation | CLIP score | Improved |
| use larger classifier-free guidance scale | Stable Diffusion · semantic violation cases | semantic fidelity | No clear effect |
Closest papers by meaning
- Grounded Text-to-Image Synthesis with Attention Refocusing (CVPR 2024 · similarity 0.93)
- CONFORM: Contrast is All You Need for High-Fidelity Text-to-Image Diffusion Models (CVPR 2024 · similarity 0.92)
- Compositional Text-to-Image Synthesis with Attention Map Control of Diffusion Models (AAAI 2024 · similarity 0.92)
- Towards Understanding Cross and Self-Attention in Stable Diffusion for Text-Guided Image Editing (CVPR 2024 · similarity 0.92)
- Noise Diffusion for Enhancing Semantic Faithfulness in Text-to-Image Synthesis (CVPR 2025 · similarity 0.92)
- Precise Parameter Localization for Textual Generation in Diffusion Models (ICLR 2025 · similarity 0.91)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
