Research map / Image generation / Fast diffusion sampling
CoInD: Enabling Logical Compositions in Diffusion Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes CoInD, a conditional diffusion training objective that adds a Fisher-divergence conditional-independence loss so that attribute marginals are independent, enabling more faithful logical compositions (AND/NOT) of attributes under non-uniform and partial support.
Problem
Conditional diffusion models trained with classifier-free guidance violate conditional independence of attribute marginals, so compositional sampling (AND, NOT over attributes) fails, especially under non-uniform or partial training support.
Approach
Adds a conditional-independence loss to the standard score-matching objective, penalizing the Fisher divergence between the joint conditional score and the product-of-marginals factorization (approximated pairwise), so the model learns accurate attribute marginals for logical composition.
Key results
- On Colored MNIST partial support, CoInD (lambda=1.0) reaches 52.38 AND conformity score vs 10.85 for LACE and 7.40 for Composed GLIDE, with JSD 1.17 vs 2.75.
- On CelebA unseen 'smiling male' joint sampling, CoInD achieves CS 8.63 and FID 43.97 vs Composed GLIDE CS 2.51 and FID 61.21.
- On Colored MNIST digit=4 generation, CoInD color entropy is 2.63 vs 1.82 for LACE and 1.71 for Composed GLIDE.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add the conditional-independence loss to score matching | Conditional diffusion models · Colored MNIST, partial support, AND | conformity score: 7.40 -> 52.38 | Helped |
| Add the conditional-independence loss to score matching | Conditional diffusion models · Shapes3D, partial support, AND | conformity score: 51.56 -> 91.10 | Helped |
| Use CoInD instead of Composed GLIDE | CelebA · Unseen smiling male | FID: 61.21 -> 43.97 | Helped |
| Increase the smile-strength parameter γ | CoInD · CelebA, smiling male | conformity score | Helped |
| Increase the smile-strength parameter γ | CoInD · CelebA, smiling male | FID | No clear effect |
| Use CoInD instead of Composed GLIDE | Colored MNIST, uniform support · Digit 4 with unconstrained color | color entropy: 1.71 -> 2.63 | Helped |
| Use CoInD instead of LACE | Colored MNIST, uniform support · Digit 4 with unconstrained color | color entropy: 1.82 -> 2.63 | Helped |
| Replace joint sampling with marginal sampling | Standard conditional diffusion · Colored MNIST, non-uniform support | conformity score: 99.98 -> 86.10 | Hurt |
| Replace joint sampling with marginal sampling | Standard conditional diffusion · Colored MNIST, partial support | conformity score: 33.14 -> 7.40 | Hurt |
| Use the theoretical upper-bound objective instead of the practical weighted-sum objective (ablation) | CoInD · Colored MNIST, partial support, AND | conformity score: 52.38 -> 23.44 | Hurt |
| Use the theoretical upper-bound objective instead of the practical weighted-sum objective (ablation) | CoInD · Shapes3D, partial support, AND | conformity score: 91.10 -> 78.13 | Hurt |
| Increase λ from 0 to a positive value (ablation) | CoInD · Colored MNIST, diagonal partial support | conformity score | Helped |
Surprising findings
- Conditional diffusion models violate conditional independence of marginals even when all attribute compositions are observed under uniform support (JSD 0.16).
- Even LACE, which trains separate per-attribute models, fails on NOT composition tasks.
Limitations
- Open-set attribute compositions outside the attribute space are not addressed.
- Large lambda causes the model to ignore input compositions, reducing conformity score.
- Relies on attribute classifiers for evaluation and on the pairwise approximation of mutual conditional independence.
Experimental setup
Datasets: Colored MNIST Shapes3D CelebA
Benchmarks: Colored MNIST Shapes3D CelebA
Metrics: Jensen-Shannon divergence conformity score R^2 FID Shannon entropy
Baselines: LACE Composed GLIDE Theoretical CoInD
Builds on: classifier-free guidance score-based generative models LACE Composed GLIDE
Closest papers by meaning
- The Emergence of Reproducibility and Consistency in Diffusion Models (ICML 2024 · similarity 0.88)
- Mechanisms of Projective Composition of Diffusion Models (ICML 2025 · similarity 0.88)
- Training Data Attribution in Diffusion Models via Mirrored Unlearning and Noise-Consistent Skew (NeurIPS 2026 · similarity 0.87)
- Sparsely Supervised Diffusion (NeurIPS 2026 · similarity 0.87)
- Towards Non-Asymptotic Convergence for Diffusion-Based Generative Models (ICLR 2024 · similarity 0.87)
- Scalable Discrete Diffusion Samplers: Combinatorial Optimization and Statistical Physics (ICLR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.