Research map / Image generation / Fast diffusion sampling
FlowDec: A flow-based full-band general audio codec with high perceptual quality
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes FlowDec, a non-adversarial 48 kHz general-audio neural codec whose DAC-based initial decoder is followed by a conditional-flow-matching stochastic postfilter that needs only 6 NFEs.
Problem
Neural audio codecs rely on adversarial GAN training for perceptual quality; the prior score-based postfilter ScoreDec is limited to speech, high bitrate (24 kbit/s), and is too slow (~60 DNN evaluations, RTF 1.7).
Approach
A joint flow-matching postfilter whose probability path starts from a Gaussian centered at the initial decoder output (y + sigma_y*eps) and interpolates linearly to the clean target with shrinking noise, giving a contractive flow field, one hyperparameter sigma_y (set by a quantile heuristic), and frequency-dependent noise levels; paired with a non-adversarial DAC variant with added CQT and L1 wav
Key results
- FlowDec-75m achieves FAD 2.53 at 6 kbit/s versus 5.23 for DAC-75, and 2.09 vs 4.15 at 7.5 kbit/s, with SIGMOS best among compared methods.
- MUSHRA listening tests (11 and 10 listeners) show no significant score differences between FlowDec and DAC at any given bitrate, with FlowDec at 4.0-7.5 kbit/s.
- Postfilter NFE reduced from ~60 (ScoreDec) to 6, giving total RTF 0.2285 for FlowDec-75 versus 1.707 for ScoreDec.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use FlowDec-75m instead of DAC-75 | 48 kHz audio codec · Test set, 4.50 kbit/s | FAD: 6.80 -> 3.01 | Helped |
| Use FlowDec-75m instead of DAC-75 | 48 kHz audio codec · Test set, 4.50 kbit/s | SI-SDR: 6.08 ± 0.18 -> 3.57 ± 0.24 | Hurt |
| Use FlowDec-25s instead of DAC-25 | 25 Hz audio codec · Test set, 4.00 kbit/s | FAD: 5.98 -> 2.47 | Helped |
| Use FlowDec-75s instead of ScoreDec | postfilter, NFE=6 · FlowDec-75s test evaluation | FAD ×100: 145.30 -> 1.62 | Helped |
| Use FlowDec instead of ScoreDec | postfilter, NFE=50 · FlowDec-75s test evaluation | FAD ×100: 5.73 -> 1.34 | Helped |
| Add CQT and waveform losses (ablation) | DAC-75 · 3.00–7.50 kbit/s | FAD: 9.68 -> 9.39; 6.80 -> 6.63; 5.23 -> 5.12; 4.15 -> 3.95 | Helped |
| Add CQT and waveform losses (ablation) | DAC-75 · 3.00–7.50 kbit/s | logSpecMSE: 80.79 ± 1.41 -> 81.93 ± 1.58; 76.95 ± 1.32 -> 77.81 ± 1.49; 74.94 ± 1.29 -> 75.65 ± 1.42; 73.05 ± 1.24 -> 73.98 ± 1.38 | Hurt |
| Set feature representation α to 0.5 (ablation) | FlowDec-75s · Test evaluation, NFE=6 | FAD: 1.62 -> 2.16 | Hurt |
| Set feature representation α to 0.5 (ablation) | FlowDec-75s · Test evaluation, NFE=6 | SIGMOS: 3.48 ± 0.03 -> 3.57 ± 0.03 | Helped |
| Use the original NCSN++ architecture (ablation) | FlowDec-75s · Test evaluation, NFE=6 | FAD: 1.62 -> 1.75 | Hurt |
| Use the original NCSN++ architecture (ablation) | FlowDec-75s · Test evaluation, NFE=6 | logSpecMSE: 80.57 ± 1.72 -> 79.84 ± 1.76 | Helped |
| Use a frequency-dependent rather than global σy (ablation) | FlowDec-75m and FlowDec-25s · 7.5 kbit/s (75 Hz) and 4.0 kbit/s (25 Hz), low NFE | FAD | Mixed |
Surprising findings
- ScoreDec produces unusable outputs at NFE=6 (SI-SDR -27.23 dB), while FlowDec achieves SI-SDR 7.55 dB at the same NFE.
- FlowDec trades lower intrusive distortion metrics (SI-SDR, logSpecMSE) for better perceptual FAD, consistent with the perception-distortion tradeoff.
Limitations
- Not streaming-capable due to noncausal NCSN++ architecture.
- Intrusive distortion metrics (SI-SDR, fwSSNR, logSpecMSE) are worse than retrained DAC.
- Training uses large internal datasets that cannot be released, limiting reproducibility.
Experimental setup
Datasets: MSP-Podcast CommonVoice 13.0 LibriTTS EARS VCTK LibriVox Expresso WavCaps-FreeSound MUSDB18-HQ MusicCaps AudioSet InternalSpeech
Benchmarks: VCTK test EARS test MUSDB18-HQ MusicCaps AudioSet
Metrics: FAD SI-SDR fwSSNR logSpecMSE SIGMOS MUSHRA-like score
Baselines: DAC-75 DAC-25 2xDAC-75 EnCodec Opus ScoreDec FlowAVSE MBD
Builds on: DAC ScoreDec SGMSE+ conditional flow matching NCSN++
Closest papers by meaning
- Flow2GAN: Hybrid Flow Matching and GAN with Multi-Resolution Network for One-/Two-step High-Fidelity Audio Generation (ICLR 2026 · similarity 0.89)
- PeriodWave: Multi-Period Flow Matching for High-Fidelity Waveform Generation (ICLR 2025 · similarity 0.86)
- DFlow: A Generative Model Combining Denoising AutoEncoder and Normalizing Flow for High Fidelity Waveform Generation (ICML 2024 · similarity 0.85)
- UniFlow-Audio: Unified Flow Matching for Audio Generation from Omni-Modalities (NeurIPS 2026 · similarity 0.85)
- Vocos: Closing the gap between time-domain and Fourier-based neural vocoders for high-quality audio synthesis (ICLR 2024 · similarity 0.85)
- Multi-band Frequency Reconstruction for Neural Psychoacoustic Coding (ICML 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.