Research map / Label-efficient and robust learning / Recognition and long-tail
CORAL: Disentangling Latent Representations in Long-Tailed Diffusion
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes CORAL, which adds a projection head on the U-Net bottleneck of a diffusion model and a time-weighted supervised contrastive loss to separate tail and head class latents, improving long-tailed class-conditional generation.
Problem
Diffusion models trained on long-tailed class distributions generate low-diversity, low-quality samples for tail classes, due to overlapping (entangled) latent class representations in the U-Net bottleneck.
Approach
Apply a supervised contrastive loss to a projection head on the denoising U-Net bottleneck features (latent space of the denoiser itself), combined with the diffusion loss via a time-dependent weight, so class-wise latents separate; the projection head is discarded at sampling time.
Key results
- On CIFAR10-LT (rho=0.01), CORAL reaches FID 5.32 vs 6.17 for DDPM and 5.62 for CBDM.
- On ImageNet-LT 64x64, CORAL achieves FID 16.11 and IS 24.17, vs DDPM FID 17.08 / IS 21.03 and CBDM FID 22.66.
- On CIFAR10 balanced, CORAL FID 3.30 vs DDPM 3.84 and CBDM 3.61.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add CORAL latent contrastive alignment | DDPM · CIFAR10-LT (ρ = 0.01) | FID: 5.62 -> 5.32 | Helped |
| Add CORAL latent contrastive alignment | DDPM · CIFAR10-LT (ρ = 0.001) | FID: 12.74 -> 11.03 | Helped |
| Add CORAL latent contrastive alignment | DDPM · CIFAR100-LT (ρ = 0.01) | FID: 6.02 -> 5.37 | Helped |
| Add CORAL latent contrastive alignment | DDPM · CelebA-5 | FID: 8.74 -> 8.12 | Helped |
| Add CORAL latent contrastive alignment | DDPM · ImageNet-LT | FID: 17.08 -> 16.11 | Helped |
| Add CORAL latent contrastive alignment | DDPM · CIFAR10 | FID: 3.61 -> 3.30 | Helped |
| Add CORAL latent contrastive alignment | DDPM · CIFAR100 | FID: 3.37 -> 2.86 | Helped |
Surprising findings
- Ambient-space contrastive regularization (DiffROP-style) degrades ImageNet-LT FID to 24.73 from DDPM's 17.08, below baseline.
- CBDM has worse FID than DDPM on ImageNet-LT (22.66 vs 17.08).
Limitations
- Added computational complexity during training relative to standard diffusion training.
- Hyperparameters (tau_SC, tau_r, omega) require per-dataset tuning.
- Evaluated only on class-conditional generation with categorical imbalance, not text-to-image models.
Experimental setup
Datasets: CIFAR10-LT CIFAR100-LT CelebA-5 ImageNet-LT CIFAR10 CIFAR100
Benchmarks: CIFAR10-LT CIFAR100-LT CelebA-5 ImageNet-LT CIFAR10 CIFAR100
Metrics: FID IS F8 F1/8 Recall (improved)
Baselines: DDPM CBDM T2H DiffROP-style ambient contrastive regularization
Builds on: DDPM classifier-free guidance SupCon CBDM T2H
Closest papers by meaning
- Learning Diffusion Models with Flexible Representation Guidance (NeurIPS 2025 · similarity 0.89)
- Nested Diffusion Models Using Hierarchical Latent Priors (CVPR 2025 · similarity 0.88)
- Probing the Latent Hierarchical Structure of Data via Diffusion Models (ICLR 2025 · similarity 0.88)
- DiffuLT: Diffusion for Long-tail Recognition Without External Knowledge (NeurIPS 2024 · similarity 0.88)
- Unlocking Dataset Distillation with Diffusion Models (NeurIPS 2025 · similarity 0.88)
- CASL: Concept-Aligned Sparse Latents for Interpreting Diffusion Models (NeurIPS 2026 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.