Research map / Image generation / Autoregressive and latent generation
Deep Voice 2: Multi-Speaker Neural Text-to-Speech
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace Deep Voice 1 with the improved Deep Voice 2 architecture | Single-speaker TTS · Single-speaker English speech database | MOS: 2.05\pm 0.24 -> 2.96\pm 0.38 | Helped |
| Replace Griffin-Lim with a WaveNet vocoder | Tacotron · Single-speaker English speech database | MOS: 2.57\pm 0.28 -> 4.17\pm 0.18 | Helped |
| Increase the Deep Voice 2 WaveNet from 20 to 80 layers (ablation) | Multi-speaker Deep Voice 2 · VCTK | MOS: 2.87$\pm$0.13 -> 3.53$\pm$0.12 | Helped |
| Replace Griffin-Lim with a WaveNet vocoder | Multi-speaker Tacotron · VCTK | MOS: 1.68$\pm$0.12 -> 2.51$\pm$0.13 | Helped |
| Replace Griffin-Lim with a WaveNet vocoder | Multi-speaker Tacotron · Audiobooks | MOS: 1.73$\pm$0.22 -> 2.11$\pm$0.20 | Helped |
| Use WaveNet rather than Griffin-Lim for Tacotron spectrogram conversion | Multi-speaker Tacotron · VCTK | speaker classification accuracy: 99.4\% -> 60.9\% | Hurt |
| Use WaveNet rather than Griffin-Lim for Tacotron spectrogram conversion | Multi-speaker Tacotron · Audiobooks | speaker classification accuracy: 93.9\% -> 66.5\% | Hurt |
| Replace Deep Voice 1 models with Deep Voice 2 models | Single-speaker Deep Voice · Single-speaker English speech database | phoneme pair error rate: 7 \% -> 0.1 \% | Helped |
| Replace Deep Voice 1 models with Deep Voice 2 models | Single-speaker Deep Voice · Single-speaker English speech database | duration mean absolute error: 38.0 ms -> 17.5 ms | Helped |
| Replace Deep Voice 1 models with Deep Voice 2 models | Single-speaker Deep Voice · Single-speaker English speech database | frequency mean absolute error: 29.4 Hz -> 24.5 Hz | Helped |
| Combine convolutional and recurrent predictions for F0 instead of using either alone (ablation) | Deep Voice 2 frequency model · Single-speaker speech synthesis | synthesis quality | Helped |
| Remove the 1×1 convolution and share the conditioner bias across WaveNet layers (ablation) | Deep Voice 2 vocal model · Speech synthesis | perceptual quality | No clear effect |
Closest papers by meaning
- Transfer Learning from Speaker Verification to Multispeaker Text-To-Speech Synthesis (NeurIPS 2018 · similarity 0.91)
- Deep Voice 3: Scaling Text-to-Speech with Convolutional Sequence Learning (ICLR 2018 · similarity 0.90)
- NaturalSpeech 2: Latent Diffusion Models are Natural and Zero-Shot Speech and Singing Synthesizers (ICLR 2024 · similarity 0.87)
- Mega-TTS 2: Boosting Prompting Mechanisms for Zero-Shot Speech Synthesis (ICLR 2024 · similarity 0.86)
- CLaM-TTS: Improving Neural Codec Language Model for Zero-Shot Text-to-Speech (ICLR 2024 · similarity 0.85)
- IndicVoices-R: Unlocking a Massive Multilingual Multi-speaker Speech Corpus for Scaling Indian TTS (NeurIPS 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.