AI Paper Atlas

Research map / Image generation / Autoregressive and latent generation

AudioLDM: Text-to-Audio Generation with Latent Diffusion Models

Haohe Liu, Zehua Chen, Yi Yuan, Xinhao Mei, Xubo Liu, Danilo Mandic, Wenwu Wang, Mark D Plumbley

ICML 2023 · Image generation › Autoregressive and latent generation · approach: Diffusion audio synthesis · 79 citations (OpenAlex)

This paper tried a common change: More diffusion sampling steps

Increase DDIM sampling steps from 10 to 100 · AudioCaps test · FD: 55.84 -> 30.17 Improved

What 457 accepted papers report for this change

See this change in detailMatched by the wording of each paper's own result, read by AI from the full text. Papers report gains more often than failures, so this shows what gets published, not the chance a change works.

Read the paper

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
Train the latent diffusion model using audio-only CLAP embeddings rather than text+audio conditions (ablation)AudioLDM-S · AudioCaps testFD: 31.26 -> 29.48Improved
Train the latent diffusion model using audio-only CLAP embeddings rather than text+audio conditions (ablation)AudioLDM-S-Full · AudioCaps testFD: 27.20 -> 23.47Improved
Increase compression level from r=4 to r=8 or r=16 (ablation)AudioLDM-S · AudioCaps testFD: 29.48 -> 33.50; 29.48 -> 34.32Hurt
Replace the full attention mechanism with one-layer multi-head self-attention (ablation)AudioLDM-S · AudioCaps testFD: 29.48 -> 33.12Hurt
Use balanced sampling (ablation)AudioLDM-S · AudioCaps testFD: 29.48 -> 34.05Hurt
Add conditioning augmentation (ablation)AudioLDM-S · AudioCaps testFD: 29.48 -> 31.88Hurt
Add conditioning augmentation (ablation)AudioLDM-S · AudioCaps testOVL: 63.41 -> 64.49Improved
Add conditioning augmentation (ablation)AudioLDM-S · AudioCaps testREL: 64.83 -> 65.01Improved
Increase DDIM sampling steps from 10 to 100 (ablation)AudioLDM-S · AudioCaps testFD: 55.84 -> 30.17Improved
Increase DDIM sampling steps from 100 to 200 (ablation)AudioLDM-S · AudioCaps testFD: 30.17 -> 29.48Improved
Increase model size from AudioLDM-S to AudioLDM-LAudioLDM · AudioCaps testFD: 29.48 -> 27.12Improved
Train with the larger collection of AudioCaps, AudioSet, FreeSound, and BBC SFXAudioLDM-L-Full · AudioCaps testFD: 27.12 -> 23.31Improved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy