Research map / Image generation / Text-to-image diffusion
Object-Conditioned Energy-Based Attention Map Alignment in Text-to-Image Diffusion Models
Yasi Zhang, Peiyu Yu, Ying Nian Wu
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add the repulsive (negative sampling) term to the binding loss (ablation) | EBAMA lambda=0 · AnE dataset Object-Object | T-C Sim.: 0.765 -> 0.823 | Improved |
| add the repulsive term to the full method (ablation) | EBAMA · AnE dataset Object-Object | T-C Sim.: 0.819 -> 0.836 | Improved |
| condition the repulsive energy on objects rather than averaging object and modifier energy (ablation) | EBAMA lambda=0 · AnE dataset Animal-Object | T-C Sim.: 0.832 -> 0.851 | Improved |
| object conditioning on Animal-Animal prompts with no modifiers (ablation) | EBAMA lambda=0 · AnE dataset Animal-Animal | T-C Sim.: 0.814 -> 0.814 | No clear effect |
| use cosine similarity energy instead of average KL divergence (ablation) | EBAMA lambda=0 · AnE dataset Object-Object | T-C Sim.: 0.817 -> 0.823 | Improved |
| add object-centric intensity regularizer (lambda>0) on harder prompts (ablation) | EBAMA · AnE dataset Object-Object | T-C Sim.: 0.823 -> 0.836 | Improved |
| set intensity weight lambda to 1.0 (ablation) | EBAMA · qualitative prompts | image quality / binding | Hurt |
| update latents at all 50 steps instead of the first 25 (ablation) | EBAMA · qualitative prompts | artifacts | Hurt |
| append more unrelated external modifiers (n=1 to 3) (ablation) | EBAMA · AnE dataset Animal-Animal | Full Sim.: 0.340 -> 0.331 | Hurt |
| replace SG-style pair loss with EBAMA binding loss | Stable Diffusion v1.4 · AnE dataset Animal-Animal | T-C Sim.: 0.767 -> 0.817 | Improved |
| use EBAMA instead of AnE for inference-time latent update | Stable Diffusion v1.4 · ABC-6K 100 prompts | generation time (min): 47.0 -> 20.9 | Improved |
Closest papers by meaning
- Text Embedding is Not All You Need: Attention Control for Text-to-Image Semantic Alignment with Text Self-Attention Maps (CVPR 2025 · similarity 0.90)
- CONFORM: Contrast is All You Need for High-Fidelity Text-to-Image Diffusion Models (CVPR 2024 · similarity 0.89)
- Information Theoretic Text-to-Image Alignment (ICLR 2025 · similarity 0.88)
- Towards Understanding Cross and Self-Attention in Stable Diffusion for Text-Guided Image Editing (CVPR 2024 · similarity 0.88)
- CoMat: Aligning Text-to-Image Diffusion Model with Image-to-Text Concept Matching (NeurIPS 2024 · similarity 0.88)
- Towards Transformer-Based Aligned Generation with Self-Coherence Guidance (CVPR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
