Research map / Image generation / Text-to-image diffusion
Predicated Diffusion: Predicate Logic-Based Attention Guidance for Text-to-Image Diffusion Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Predicated Diffusion represents prompt meanings as predicate-logic propositions, converts them to differentiable fuzzy-logic losses on cross-attention maps, and uses them as guidance for Stable Diffusion to fix missing objects, attribute leakage, and possession failure.
Problem
Text-to-image diffusion models often fail to capture the intended meaning of prompts: objects go missing, objects mix, adjectives leak onto wrong objects, and possession relations (e.g., holding) are ignored.
Approach
Treat cross-attention map pixel intensities as fuzzy truth values of predicates, express prompt semantics as first-order predicate logic propositions (existence, modification, one-to-one correspondence, possession), and use the negative log of the resulting fuzzy truth as a training-free guidance loss during denoising.
Key results
- In concurrent existence, Predicated Diffusion reduced missing-object rate to 18.5/28.5 (lenient/strict) vs 25.3/36.3 for Attend-and-Excite and 54.7/66.0 for Stable Diffusion, and achieved best CLIP-IQA (0.775).
- In one-to-one correspondence, attribute leakage fell to 44.8% from SynGen's 36.8%? No: Predicated Diffusion attained 10.0/16.5 missing objects and best CLIP-IQA 0.769, with leakage 33.0 (lowest among methods reported).
- In possession experiments, Predicated Diffusion reduced missing objects to 4.0/7.0 and possession failure to 29.5% vs 52.5% for Stable Diffusion.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Apply predicate-logic attention guidance for concurrent object existence | Stable Diffusion v1.4 · Concurrent Existence | Missing objects (lenient / strict, %): 54.7 / 66.0 -> 18.5 / 28.5 | Helped |
| Apply predicate-logic attention guidance for concurrent object existence | Stable Diffusion v1.4 · Concurrent Existence | CLIP-IQA: 0.761 -> 0.775 | Helped |
| Apply predicate-logic attention guidance with one-to-one attribute correspondence | Stable Diffusion v1.4 · One-to-One Correspondence | Attribute leakage (%): 88.5 -> 33.0 | Helped |
| Apply predicate-logic attention guidance with one-to-one attribute correspondence | Stable Diffusion v1.4 · One-to-One Correspondence | Text-image similarity: 0.345 -> 0.379 | Helped |
| Apply predicate-logic attention guidance for possession relationships | Stable Diffusion v1.4 · Possession | Possession failure (%): 52.5 -> 29.5 | Helped |
| Apply predicate-logic attention guidance for possession relationships | Stable Diffusion v1.4 · Possession | Fidelity (%): 33.5 -> 52.0 | Helped |
Surprising findings
- Attend-and-Excite, despite preventing missing objects, increased possession failure relative to Stable Diffusion and lowered human-rated fidelity.
- SynGen degraded CLIP-IQA image quality compared with its Stable Diffusion backbone.
Limitations
- Inherits diffusion model limitations such as inaccurate object counting and bias toward typical examples; the one-to-one loss fails for multiple instances of the same noun with different adjectives.
- Only unary predicates are handled; relations such as Above(x, y) are left to future work.
- Guidance is applied only to the first 25 of 50 denoising steps and depends on Stable Diffusion v1.4 as backbone.
Experimental setup
Datasets: ABC-6K
Benchmarks: Custom prompt sets for concurrent existence (400 prompts) one-to-one correspondence (400 prompts) possession (10 prompts x 20 images) ABC-6K prompts (qualitative)
Metrics: missing objects rate attribute leakage rate possession failure rate human fidelity CLIP text-image similarity BLIP text-text similarity CLIP-IQA
Baselines: Stable Diffusion Composable Diffusion Structure Diffusion Attend-and-Excite SynGen
Builds on: Attend-and-Excite Composable Diffusion SynGen Structure Diffusion classifier-free guidance
Closest papers by meaning
- CONFORM: Contrast is All You Need for High-Fidelity Text-to-Image Diffusion Models (CVPR 2024 · similarity 0.89)
- Devil is in the Detail: Towards Injecting Fine Details of Image Prompt in Image Generation via Conflict-free Guidance and Stratified Attention (CVPR 2025 · similarity 0.88)
- The Hidden Language of Diffusion Models (ICLR 2024 · similarity 0.88)
- Prompt-Free Diffusion: Taking "Text" out of Text-to-Image Diffusion Models (CVPR 2024 · similarity 0.88)
- Diffusion PID: Interpreting Diffusion via Partial Information Decomposition (NeurIPS 2024 · similarity 0.87)
- Prompting Hard or Hardly Prompting: Prompt Inversion for Text-to-Image Diffusion Models (CVPR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.