Research map / Image generation / Fast diffusion sampling
DFlow: A Generative Model Combining Denoising AutoEncoder and Normalizing Flow for High Fidelity Waveform Generation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes DFlow, a waveform vocoder that jointly trains a normalizing flow and a denoising autoencoder (auxiliary AR flow as DAE encoder, volume-preserving primary flow, UNet decoder), plus a scaled DFlow-XL trained on LibriTTS.
Problem
Standard normalizing-flow neural vocoders lag GAN and diffusion vocoders in perceptual quality and are sensitive to small input variations and initial errors, producing periodic artifacts.
Approach
Combine an NF with a DAE trained end-to-end: an invertible autoregressive flow serves as both the DAE encoder and part of the prior flow, a volume-preserving primary flow is inverted at sampling, and a feed-forward UNet decoder reconstructs clean audio from noise-injected inputs with an MAE loss.
Key results
- On LJSpeech, DFlow achieves MOS 4.42 (ground truth 4.53), above HiFi-GAN 4.39, WaveGlow 3.92 and WaveNet 4.08, with PESQ 3.43 and MCD 0.71.
- DFlow runs at GPU RTF 0.014 (about 71x faster than real time on one V100), roughly 3x faster than WaveGlow and the fastest likelihood model reported.
- On LibriTTS, DFlow-XL reaches IND MOS 4.30, matching BigVGAN 4.30, and OOD non-speech MOS 4.06 vs BigVGAN 4.04.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use DFlow instead of HiFi-GAN | Waveform generation · LJ-Speech test set | MOS: 4.39±0.08 -> 4.42±0.06 | Helped |
| Use DFlow instead of HiFi-GAN | VCTK vocoding · VCTK IND | MOS: 4.34±0.06 -> 4.39±0.07 | Helped |
| Use DFlow instead of HiFi-GAN | VCTK vocoding · VCTK OOD speech | MOS: 3.94±0.08 -> 4.05±0.09 | Helped |
| Use DFlow instead of HiFi-GAN | VCTK vocoding · VCTK OOD non-speech | MOS: 3.84±0.08 -> 4.02±0.08 | Helped |
| Scale up from DFlow-L to DFlow-XL | DFlow universal vocoder · LibriTTS OOD non-speech | MOS: 3.91±0.08 -> 4.06±0.11 | Helped |
| Add noise injection (ablation) | DFlow · LJ-Speech test set | CMOS: -0.25 | Helped |
| Add AR Flow on Auxiliary Flow f (ablation) | DFlow · LJ-Speech test set | CMOS: -0.19 | Helped |
| Keep the invertibility constraint on Auxiliary Flow f (ablation) | DFlow · LJ-Speech test set | Training convergence | Helped |
| Keep the volume-preserving constraint on Primary Flow g (ablation) | DFlow · LJ-Speech test set | CMOS: -0.18 | Helped |
| Use the DFlow structure instead of a standard NF (ablation) | Waveform generation · LJ-Speech test set | CMOS: -0.18 | Helped |
Surprising findings
- Removing noise injection makes DFlow perform worse than the standard NF model (CMOS -0.25 vs -0.18).
- Without the invertible constraint on the auxiliary flow, training fails to converge.
Limitations
- Periodic activation functions such as SnakeBeta were not applied, leaving DFlow behind BigVGAN on some metrics.
- Objective metrics show HiFi-GAN best on M-STFT, and DFlow is not best on all objectives.
- Large-scale results are only on LibriTTS; DFlow-XL is only on par with BigVGAN rather than clearly better.
Experimental setup
Datasets: LJSpeech VCTK LibriTTS MUSDB18-HQ Multilingual TEDx
Benchmarks: LJSpeech VCTK LibriTTS
Metrics: MOS CMOS M-STFT PESQ MCD periodicity error V/UV F1 RTF FLOPs
Baselines: WaveNet (MoL) WaveGlow HiFi-GAN DiffWave BigVGAN Big-HiFi-GAN Standard NF
Builds on: WaveNet WaveGlow variational autoencoder inverse autoregressive flow dequantization
Closest papers by meaning
- Generative Pre-training for Speech with Flow Matching (ICLR 2024 · similarity 0.86)
- MDSGen: Fast and Efficient Masked Diffusion Temporal-Aware Transformers for Open-Domain Sound Generation (ICLR 2025 · similarity 0.86)
- OmniFlow: Any-to-Any Generation with Multi-Modal Rectified Flows (CVPR 2025 · similarity 0.86)
- CSSinger: End-to-End Chunkwise Streaming Singing Voice Synthesis System Based on Conditional Variational Autoencoder (AAAI 2025 · similarity 0.85)
- DiffAR: Denoising Diffusion Autoregressive Model for Raw Speech Waveform Generation (ICLR 2024 · similarity 0.85)
- FlowDec: A flow-based full-band general audio codec with high perceptual quality (ICLR 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.