Research map / Image generation / Autoregressive and latent generation
NaturalSpeech : End-to-End Text-to-Speech Synthesis With Human-Level Quality
Xu Tan, Jiawei Chen, Haohe Liu, Jian Cong, Chen Zhang, Yanqing Liu, Xi Wang, Yichong Leng, Yuanhao Yi, Lei He, Sheng Zhao, Tao Qin and 2 more
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add phoneme pre-training (ablation) | NaturalSpeech · LJSpeech | CMOS: -0.09 CMOS drop when removed | Improved |
| add differentiable durator (ablation) | NaturalSpeech · LJSpeech | CMOS: -0.12 CMOS drop when removed | Improved |
| add bidirectional prior/posterior module (ablation) | NaturalSpeech · LJSpeech | CMOS: -0.09 CMOS drop when removed | Improved |
| add memory mechanism in VAE (ablation) | NaturalSpeech · LJSpeech | CMOS: -0.06 CMOS drop when removed | Improved |
| end-to-end NaturalSpeech instead of VITS | TTS system · LJSpeech | CMOS: -0.20 -> 0 | Improved |
| end-to-end NaturalSpeech instead of FastSpeech 2 + HiFiGAN | TTS system · LJSpeech | MOS: 4.32 -> 4.56 | Improved |
| use phoneme pre-training in FastSpeech 2 pipeline | FastSpeech 2 + HiFiGAN · LJSpeech | CMOS: 0.12 CMOS gain | Improved |
| use predicted rather than ground-truth pitch/duration | FastSpeech 2 + HiFiGAN · LJSpeech | CMOS: -0.14 | Hurt |
| use mel decoder output instead of ground-truth mel | FastSpeech 2 + HiFiGAN · LJSpeech | CMOS: -0.15 | Hurt |
| use vocoder on ground-truth mel instead of recordings | HiFiGAN · LJSpeech | CMOS: -0.04 | Hurt |
| reduce Grad-TTS inference steps from 1000 to 10 | Grad-TTS + HiFiGAN · LJSpeech | RTF: 4.120 -> 0.082 | Improved |
| keep strange-rhythm recordings in evaluation | NaturalSpeech · LJSpeech | CMOS: -0.01 -> +0.09 | Improved |
Closest papers by meaning
- NaturalSpeech 2: Latent Diffusion Models are Natural and Zero-Shot Speech and Singing Synthesizers (ICLR 2024 · similarity 0.89)
- Audio Large Language Models Can Be Descriptive Speech Quality Evaluators (ICLR 2025 · similarity 0.87)
- Scaling Speech-Text Pre-training with Synthetic Interleaved Data (ICLR 2025 · similarity 0.87)
- TTA-Bench: A Comprehensive Benchmark for Evaluating Text-to-Audio Models (AAAI 2026 · similarity 0.87)
- A Variational Framework for Improving Naturalness in Generative Spoken Language Models (ICML 2025 · similarity 0.86)
- DELTA-TTS: Adapting Autoregressive Model into a Diffusion Language Model for Text-to-Speech (NeurIPS 2026 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
