Research map / Image generation / Autoregressive and latent generation
AudioLDM: Text-to-Audio Generation with Latent Diffusion Models
Haohe Liu, Zehua Chen, Yi Yuan, Xinhao Mei, Xubo Liu, Danilo Mandic, Wenwu Wang, Mark D Plumbley
This paper tried a common change: More diffusion sampling steps
Increase DDIM sampling steps from 10 to 100 · AudioCaps test · FD: 55.84 -> 30.17 Improved
What 457 accepted papers report for this change
- 277 improved
- 77 no clear effect
- 26 mixed
- 77 hurt
- Optimality of FSQ Tokens for Continuous Diffusion for Categorical Data with Application to Text-to-Speech ICML 2026 · increase denoising steps from 5 to 25 for FSQ-original · SEED-TTS test-en · WER: 2.39% -> 2.00%Improved
- DiscoForcing: A Unified Framework for Real-Time Audio-Driven Character Control with Diffusion Forcing ICML 2026 · increase denoising steps from 10 to 100 · AIST++ · latency (ms/frame): 26.26 -> 261.91Hurt
- Variable-Length Tokenization via Learnable Global Merging for Diffusion Transformers ICML 2026 · more diffusion sampling steps (25 to 100) for our method · ImageNet 256x256, 32 tokens · gFID: 2.89 -> 2.54Improved
See this change in detailMatched by the wording of each paper's own result, read by AI from the full text. Papers report gains more often than failures, so this shows what gets published, not the chance a change works.
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Train the latent diffusion model using audio-only CLAP embeddings rather than text+audio conditions (ablation) | AudioLDM-S · AudioCaps test | FD: 31.26 -> 29.48 | Improved |
| Train the latent diffusion model using audio-only CLAP embeddings rather than text+audio conditions (ablation) | AudioLDM-S-Full · AudioCaps test | FD: 27.20 -> 23.47 | Improved |
| Increase compression level from r=4 to r=8 or r=16 (ablation) | AudioLDM-S · AudioCaps test | FD: 29.48 -> 33.50; 29.48 -> 34.32 | Hurt |
| Replace the full attention mechanism with one-layer multi-head self-attention (ablation) | AudioLDM-S · AudioCaps test | FD: 29.48 -> 33.12 | Hurt |
| Use balanced sampling (ablation) | AudioLDM-S · AudioCaps test | FD: 29.48 -> 34.05 | Hurt |
| Add conditioning augmentation (ablation) | AudioLDM-S · AudioCaps test | FD: 29.48 -> 31.88 | Hurt |
| Add conditioning augmentation (ablation) | AudioLDM-S · AudioCaps test | OVL: 63.41 -> 64.49 | Improved |
| Add conditioning augmentation (ablation) | AudioLDM-S · AudioCaps test | REL: 64.83 -> 65.01 | Improved |
| Increase DDIM sampling steps from 10 to 100 (ablation) | AudioLDM-S · AudioCaps test | FD: 55.84 -> 30.17 | Improved |
| Increase DDIM sampling steps from 100 to 200 (ablation) | AudioLDM-S · AudioCaps test | FD: 30.17 -> 29.48 | Improved |
| Increase model size from AudioLDM-S to AudioLDM-L | AudioLDM · AudioCaps test | FD: 29.48 -> 27.12 | Improved |
| Train with the larger collection of AudioCaps, AudioSet, FreeSound, and BBC SFX | AudioLDM-L-Full · AudioCaps test | FD: 27.12 -> 23.31 | Improved |
Closest papers by meaning
- Generative Audio Language Modeling with Continuous-valued Tokens and Masked Next-Token Prediction (ICML 2025 · similarity 0.90)
- AudioCALM: Continuous Autoregressive Language Modeling for Universal Audio Generation (NeurIPS 2026 · similarity 0.90)
- Fast Timing-Conditioned Latent Audio Diffusion (ICML 2024 · similarity 0.89)
- ETTA: Elucidating the Design Space of Text-to-Audio Models (ICML 2025 · similarity 0.89)
- MATS: An Audio Language Model under Text-only Supervision (ICML 2025 · similarity 0.89)
- SoundCTM: Unifying Score-based and Consistency Models for Full-band Text-to-Sound Generation (ICLR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
