Research map / Image generation / Image restoration
Complex-Cycle-Consistent Diffusion Model for Monaural Speech Enhancement
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SEDM, a phase-aware diffusion model for monaural speech enhancement that replaces Gaussian noise with real-world DEMAND noise clips in the forward process and uses a complex-cycle-consistent (CCC) loss between magnitude and phase estimates.
Problem
Monaural speech enhancement with diffusion models typically only estimates magnitude (keeping noisy phase), uses Gaussian noise in the diffusion process, and ignores the intrinsic relationship between magnitude and phase.
Approach
Two diffusion networks estimate magnitude and phase separately; the forward process adds real-world noise clips instead of Gaussian noise; a noise-aware reverse process estimates both speech and noise spectra; and a CCC module with two LSTMs maps magnitude to phase and phase to magnitude, adding cycle losses to train the networks.
Key results
- SEDM-L reaches STOI 80.2%, PESQ 2.75 and fwSNRseg 14.30 dB on IEEE, the best among compared methods including SCP-GAN (77.3%, 2.66, 14.04).
- On VCTK, SEDM-L achieves PESQ 3.59 and SSNR 11.8, versus SCP-GAN's 3.52 and 10.8.
- Ablation on IEEE shows PESQ rising from 2.21 (baseline) to 2.75 with the full model (diffusion + phase-aware + CCC).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add diffusion modeling (ablation) | SEDM · IEEE | PESQ: 2.21 -> 2.43 | Helped |
| Add phase-aware estimation (ablation) | SEDM · IEEE | PESQ: 2.21->2.33 | Helped |
| Add CCC to phase-aware estimation (ablation) | SEDM without diffusion · IEEE | PESQ: 2.33 -> 2.60 | Helped |
| Add CCC to diffusion with phase-aware estimation (ablation) | SEDM · IEEE | PESQ: 2.55 -> 2.75 | Helped |
| Add diffusion to phase-aware estimation with CCC (ablation) | SEDM · IEEE | PESQ: 2.60 -> 2.75 | Helped |
| Combine estimated magnitude with phase loss and estimated phase with magnitude loss (ablation) | Speech enhancement · IEEE | PESQ: 1.52 -> 1.71 | Helped |
| Use real-world noise instead of Gaussian noise | SEDM · VCTK and DNS challenge | speech enhancement performance | Mixed |
| Evaluate with unseen rather than seen real-world noise types | SEDM · VCTK and DNS challenge | speech enhancement performance | Hurt |
| Use SEDM-L instead of SCP-GAN | Speech enhancement · VCTK | CBAK: 4.25 -> 4.22 | Hurt |
| Use SEDM-L instead of RemixIT | Speech enhancement · DNS challenge without reverberations | WP: 2.95 -> 2.93 | Hurt |
| Use SEDM-L instead of RemixIT | Speech enhancement · DNS challenge without reverberations | SI-SDR: 19.7 -> 13.2 | Hurt |
| Use SEDM-L instead of NASE | Speech enhancement · VCTK + DEMAND | PESQ: 3.01 -> 3.59 | Helped |
Surprising findings
- Models trained with real-world noise are initially inferior to Gaussian-noise models in shallower networks, and only become competitive as depth increases.
- On the DNS challenge, SEDM-L does not beat RemixIT on wide-band PESQ (2.93 vs 2.95), and the paper describes its performance as competitive rather than superior.
Limitations
- Performance degrades on unseen noise types due to noise domain mismatch.
- Shallow real-world-noise models underperform Gaussian-noise models; advantage only emerges at greater depth.
- Competitor models were reproduced by the authors under a shared setup, which may affect comparability.
Experimental setup
Datasets: IEEE TIMIT VCTK DEMAND DNS challenge
Benchmarks: IEEE TIMIT VCTK DNS challenge
Metrics: STOI PESQ fwSNRseg CSIG CBAK COVL SSNR WP NP SI-SDR
Baselines: DCTCRN RemixIT FRCRN CMGAN SCP-GAN DDPM NU-Wave DiffuSE CDiffuSE SGMSE NASE
Builds on: DDPM NASE SGMSE CMGAN FRCRN
Closest papers by meaning
- Shallow Diffusion for Fast Speech Enhancement (Student Abstract) (AAAI 2024 · similarity 0.88)
- BSDB-Net: Band-Split Dual-Branch Network with Selective State Spaces Mechanism for Monaural Speech Enhancement (AAAI 2025 · similarity 0.88)
- DiffAR: Denoising Diffusion Autoregressive Model for Raw Speech Waveform Generation (ICLR 2024 · similarity 0.85)
- Cauchy Diffusion: A Heavy-tailed Denoising Diffusion Probabilistic Model for Speech Synthesis (AAAI 2025 · similarity 0.85)
- Predictive–Generative Drift Decomposition for Speech Enhancement and Separation (NeurIPS 2026 · similarity 0.84)
- DiTAR: Diffusion Transformer Autoregressive Modeling for Speech Generation (ICML 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.