Research map / Image generation / Fast diffusion sampling
Cauchy Diffusion: A Heavy-tailed Denoising Diffusion Probabilistic Model for Speech Synthesis
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Cauchy Diffusion, a DDPM-based neural vocoder that replaces Gaussian noise with heavy-tailed Cauchy noise (constructed as a ratio of two Gaussian diffusion processes) to improve prosody diversity in speech synthesis.
Problem
DDPM-based neural vocoders suffer mode collapse on imbalanced speech data, which limits modeling of prosody diversity in synthesized speech.
Approach
Build a Cauchy diffusion process from the ratio of two Gaussian diffusion processes, derive a schedule for Cauchy prior and posterior scales, train with a KL loss between zero-location Cauchy distributions with a predicted squared scale, and sample with a DDIM-style stochastic/deterministic procedure.
Key results
- On LJSpeech, Cauchy Diffusion (η=1) reaches PESQ 4.055, STOI 0.986 and the best MCD of 1.862 (vs. HiFiGAN 2.136), with MOS 4.00 vs. 3.92 for HiFiGAN.
- On VCTK, Cauchy Diffusion (η=1) achieves the best MCD of 1.421 (vs. UnivNet 1.520) but a lower MOS of 3.76 vs. 3.80 for HiFiGAN and 3.85 for WaveGlow.
- Ablation on LJSpeech shows the tradeoff coefficient of 10 gives the best MCD, a coefficient 1000x larger than in prior work.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace Gaussian diffusion noise with Cauchy noise | DDPM neural vocoder · LJSpeech | MCD: 2.062 ± 0.521 -> 1.862 ± 0.443 | Helped |
| replace Gaussian diffusion noise with Cauchy noise | DDPM neural vocoder · VCTK | MCD: 1.520 ± 0.654 -> 1.421 ± 0.606 | Helped |
| use stochastic rather than deterministic sampling (η = 1) (ablation) | Cauchy Diffusion · LJSpeech | MCD: 1.991 ± 0.486 -> 1.862 ± 0.443 | Helped |
| use stochastic rather than deterministic sampling (η = 1) (ablation) | Cauchy Diffusion · VCTK | MCD: 1.493 ± 0.616 -> 1.421 ± 0.606 | Helped |
| set tradeoff coefficient λ to 10 (ablation) | Cauchy Diffusion · LJSpeech | MCD: 1.938 ± 0.477 -> 1.929 ± 0.461 | Helped |
| use T = 1000 instead of T = 50 diffusion steps (ablation) | Cauchy Diffusion · LJSpeech | PESQ: 3.994 ± 0.110 -> 3.978 ± 0.100 | No clear effect |
| increase denoiser capacity from U-net1 to U-net2 (ablation) | Cauchy Diffusion · LJSpeech | PESQ: 3.978 ± 0.100 -> 4.055 ± 0.081 | Helped |
| increase the Cauchy noise clamp value (NCV) (ablation) | Cauchy Diffusion · LJSpeech | PESQ | Hurt |
| use stochastic sampling instead of deterministic sampling at NCV 10 or 15 (ablation) | Cauchy Diffusion · LJSpeech | PESQ | Hurt |
| use stochastic sampling instead of deterministic sampling at NCV 5 (ablation) | Cauchy Diffusion · LJSpeech | PESQ | Helped |
| increase tradeoff coefficient λ across tested settings (ablation) | Cauchy Diffusion · LJSpeech | PESQ: 4.013 ± 0.082; 4.019 ± 0.072; 4.014 ± 0.110; 4.018 ± 0.080 | No clear effect |
Surprising findings
- Cauchy Diffusion with deterministic sampling already beats other neural vocoders on prosody diversity (MCD), and stochastic sampling improves it further.
- Increasing model capacity (U-net2) with heavy-tailed noise gave better performance with fewer training steps (3M vs 10M).
Limitations
- Does not support fast sampling, which is critical for real-world deployment.
- On VCTK, MOS is lower than several baselines (3.76 vs up to 3.85) and STOI is below WaveGlow's 0.960.
- Diversity is measured only indirectly via MCD, a proxy for prosody diversity.
Experimental setup
Datasets: LJSpeech VCTK
Benchmarks: LJSpeech VCTK
Metrics: PESQ STOI MCD MOS
Baselines: WaveGlow HiFiGAN UnivNet WaveGrad DiffWave PriorGrad FastDiff
Builds on: DDPM DiffWave DDIM PriorGrad location-variable convolution
Closest papers by meaning
- DiffAR: Denoising Diffusion Autoregressive Model for Raw Speech Waveform Generation (ICLR 2024 · similarity 0.90)
- DDDM-VC: Decoupled Denoising Diffusion Models with Disentangled Representation and Prior Mixup for Verified Robust Voice Conversion (AAAI 2024 · similarity 0.88)
- Shallow Diffusion for Fast Speech Enhancement (Student Abstract) (AAAI 2024 · similarity 0.87)
- DMOSpeech: Direct Metric Optimization via Distilled Diffusion Model in Zero-Shot Speech Synthesis (ICML 2025 · similarity 0.87)
- NaturalSpeech 2: Latent Diffusion Models are Natural and Zero-Shot Speech and Singing Synthesizers (ICLR 2024 · similarity 0.87)
- NaturalSpeech 3: Zero-Shot Speech Synthesis with Factorized Codec and Diffusion Models (ICML 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.