Research map / Image generation / Fast diffusion sampling
Don't Drop Your Samples! Coherence-Aware Training Benefits Conditional Diffusion
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Coherence-Aware Diffusion (CAD), which conditions a diffusion model on a per-sample coherence score alongside the condition so it can learn from noisy pairs without filtering, plus a coherence-aware classifier-free guidance variant.
Problem
Conditional diffusion models trained on noisy or misaligned conditions (web-scraped captions, noisy class labels, imperfect segmentation maps) either learn from them poorly or must discard large fractions of data via filtering.
Approach
Attach a per-sample coherence score in [0,1] to the condition as an extra model input so the network learns how much to rely on the condition, retaining all samples; at inference the coherence can be set manually, and CFG is redefined to contrast coherence 1 vs 0 instead of dropping conditions.
Key results
- On COCO-10K text-to-image, CAD achieves FID 69.4 versus 91.9 for the baseline and 85.8 for filtering, with CLIPScore 26.16 comparable to baseline 25.96.
- In a user study with 36 users, CAD images were preferred for quality in 95% of cases and for prompt alignment in 89% of cases against the baselines.
- On ImageNet class-conditional generation with label noise beta=0.5, CAD improves accuracy to 0.327 versus 0.168 for the baseline and FID 9.11 versus 14.38.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| condition on coherence during training instead of using baseline training | RIN · COCO-10K | FID: 91.9 -> 69.4 | Helped |
| condition on coherence during training instead of filtering low-coherence samples | RIN · COCO-10K | FID: 85.8 -> 69.4 | Helped |
| weight the diffusion loss by coherence instead of using CAD | RIN · COCO-10K | FID: 69.4 -> 98.3 | Hurt |
| condition on coherence during training instead of baseline training | RIN · ImageNet | FID: 14.38 -> 9.11 | Helped |
| condition on coherence during training instead of baseline training | RIN · ImageNet | Acc: 0.168 -> 0.327 | Helped |
| condition on coherence during training instead of filtering samples below coherence 0.5 | RIN · CIFAR-10 | FID: 9.48 -> 4.75 | Helped |
| condition on coherence during training instead of filtering samples below coherence 0.5 | RIN · CIFAR-10 | Acc: 0.634 -> 0.906 | Helped |
| add coherence maps to semantic-map conditioning | Stable Diffusion + ControlNet · ADE20K | FID: 33.67 -> 30.88 | Helped |
| add coherence maps to semantic-map conditioning | Stable Diffusion + ControlNet · MS COCO | FID: 20.1 -> 18.1 | Helped |
| increase the inference coherence score | CAD text-to-image model · COCO-10K | CLIPScore | Helped |
| increase the inference coherence score | CAD text-to-image model · COCO-10K | FID | Helped |
| increase coherence-aware guidance rate | CAD class-conditional model · ImageNet | Acc | Mixed |
Surprising findings
- Filtering low-coherence samples on CIFAR-10 dramatically degraded performance, with FID worse than the unfiltered baseline.
Limitations
- Performance depends on the reliability of the coherence score extraction; unreliable scores bias the model.
- Coherence scores for text rely on CLIP-based estimates, which may carry CLIP biases (e.g., emphasis on salient subjects over background).
Experimental setup
Datasets: CC12M LAION Aesthetics 6+ ImageNet CIFAR-10 ADE20K COCO
Benchmarks: COCO-10K ImageNet CIFAR-10 ADE20K MS COCO
Metrics: FID CLIPScore precision recall density coverage accuracy Inception Score mIoU user preference
Baselines: Baseline (no coherence) Filtered Weighted CAD-free RIN baseline
Builds on: classifier-free guidance RIN ControlNet Stable Diffusion
Closest papers by meaning
- Label-Noise Robust Diffusion Models (ICLR 2024 · similarity 0.88)
- Stochastic Conditional Diffusion Models for Robust Semantic Image Synthesis (ICML 2024 · similarity 0.88)
- Random Conditioning for Diffusion Model Compression with Distillation (CVPR 2025 · similarity 0.88)
- APT: Adaptive Personalized Training for Diffusion Models with Limited Data (CVPR 2025 · similarity 0.87)
- CADS: Unleashing the Diversity of Diffusion Models through Condition-Annealed Sampling (ICLR 2024 · similarity 0.87)
- Membership Inference on Text-to-Image Diffusion Models via Conditional Likelihood Discrepancy (NeurIPS 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.