Research map / Image generation / Autoregressive and latent generation
Discrete Contrastive Diffusion for Cross-Modal Music and Image Generation
Ye Zhu, Yu Wu, Kyle Olszewski, Jian Ren, Sergey Tulyakov, Yan Yan
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add step-wise parallel diffusion with intra-sample negatives (ablation) | Vanilla dance-to-music diffusion · AIST++ | Beats Coverage: 89.0$\pm$1.1 -> 93.9$\pm$}1.2 | Improved |
| Add step-wise parallel diffusion with intra-sample negatives (ablation) | Vanilla dance-to-music diffusion · AIST++ | Beats Hit: 83.8$\pm$1.5 -> 90.7$\pm$1.5 | Improved |
| Add sample-wise auxiliary diffusion with inter-sample negatives (ablation) | Vanilla dance-to-music diffusion · AIST++ | Genre Accuracy: 25.3$\pm$0.8 -> 27.2$\pm$0.5 | Improved |
| Add CDCD contrastive training (ablation) | Dance-to-music diffusion · AIST++ | Quality MOS: 3.6 -> 3.6 | No clear effect |
| Add step-wise parallel diffusion with intra-sample negatives (ablation) | Vanilla dance-to-music diffusion · TikTok Dance-Music | Beats Hit: 81.4 -> 86.3 | Improved |
| Use inter-sample negatives instead of intra-sample negatives in step-wise diffusion (ablation) | Step-wise parallel diffusion · AIST++ | Beats Coverage: 93.9$\pm$}1.2 -> 92.4$\pm$1.0 | Hurt |
| Use intra-sample negatives instead of inter-sample negatives in sample-wise diffusion (ablation) | Sample-wise auxiliary diffusion · AIST++ | Genre Accuracy: 27.2$\pm$0.5 -> 26.0$\pm$0.8 | Hurt |
| Add sample-wise auxiliary diffusion with inter-sample negatives (ablation) | VQ-D text-to-image diffusion · CUB-200 | FID: 14.61 -> 12.61 | Improved |
| Add sample-wise auxiliary diffusion with inter-sample negatives (ablation) | VQ-D text-to-image diffusion · MSCOCO | CLIPScore: 65.53 -> 66.79 | Improved |
| Use sample-wise auxiliary diffusion with inter-sample negatives rather than step-wise parallel diffusion with intra-sample negatives (ablation) | CDCD text-to-image diffusion · CUB-200 | FID: 12.73 -> 12.61 | Improved |
| Use sample-wise auxiliary diffusion with inter-sample negatives rather than step-wise parallel diffusion with intra-sample negatives (ablation) | CDCD text-to-image diffusion · MSCOCO | FID: 32.25 -> 28.76 | Improved |
| Apply CDCD contrastive diffusion | Vanilla diffusion · Dance-to-music AIST++ | Diffusion steps to convergence: 120 -> 80; ~35% | Improved |
Closest papers by meaning
- NoiseCtrl: A Sampling-Algorithm-Agnostic Conditional Generation Method for Diffusion Models (CVPR 2025 · similarity 0.88)
- Diffuse Everything: Multimodal Diffusion Models on Arbitrary State Spaces (ICML 2025 · similarity 0.88)
- A Versatile Diffusion Transformer with Mixture of Noise Levels for Audiovisual Generation (NeurIPS 2024 · similarity 0.88)
- DITTO: Diffusion Inference-Time T-Optimization for Music Generation (ICML 2024 · similarity 0.88)
- Simple Distillation for One-Step Diffusion Models (NeurIPS 2025 · similarity 0.88)
- Towards General Modality Translation with Contrastive and Predictive Latent Diffusion Bridge (NeurIPS 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
