Research map / Label-efficient and robust learning / Recognition and long-tail
Sample Efficient Adaptive Text-to-Speech
Yutian Chen, Yannis Assael, Brendan Shillingford, David Budden, Scott Reed, Heiga Zen, Quan Wang, Luis C. Cobo, Andrew Trask, Ben Laurie, Caglar Gulcehre, Aäron van den Oord and 2 more
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Fine-tune the full WaveNet after fitting the speaker embedding, rather than fitting only the embedding | SEA-All versus SEA-Emb · LibriSpeech and VCTK | naturalness MOS: 3.86 ± 0.07 -> 3.94 ± 0.08; 3.95 ± 0.07 -> 4.13 ± 0.06; 3.81 ± 0.07 -> 3.92 ± 0.07; 3.82 ± 0.07 -> 3.92 ± 0.07 | Improved |
| Fine-tune the full WaveNet after fitting the speaker embedding, rather than fitting only the embedding | SEA-All versus SEA-Emb · VCTK, <10m adaptation | voice similarity MOS: 3.18 ± 0.10 -> 3.97 ± 0.09 | Improved |
| Use an audio-conditioned embedding encoder instead of fitting the embedding from adaptation data | SEA-Enc versus SEA-Emb · LibriSpeech and VCTK | voice similarity MOS: 3.42 ± 0.10 -> 2.47 ± 0.09; 3.56 ± 0.10 -> 2.59 ± 0.09; 3.07 ± 0.10 -> 2.07 ± 0.08; 3.18 ± 0.10 -> 2.19 ± 0.09 | Hurt |
| Increase adaptation data beyond 10 seconds | SEA-All · LibriSpeech | naturalness MOS: 3.94 ± 0.08 -> 4.13 ± 0.06 | Improved |
| Increase adaptation data beyond 10 seconds | SEA-All · VCTK | naturalness MOS: 3.92 ± 0.07 -> 3.92 ± 0.07 | No clear effect |
| Increase adaptation data from 10 seconds to <5m | SEA-All · LibriSpeech | speaker verification EER: 3.17 -> 1.85 | Improved |
| Increase adaptation data from 10 seconds to <10m | SEA-All · VCTK | speaker verification EER: 7.34 -> 4.33 | Improved |
| Use embedding-only fitting instead of the learned encoder | SEA-Emb versus SEA-Enc · LibriSpeech and VCTK | speaker verification EER: 9.42 -> 2.74; 25.23 -> 10.24 | Improved |
Closest papers by meaning
- Enhancing Zero-Shot Multi-Speaker TTS with Negated Speaker Representations (AAAI 2024 · similarity 0.87)
- E-BATS: Efficient Backpropagation-Free Test-Time Adaptation for Speech Foundation Models (NeurIPS 2025 · similarity 0.85)
- DELTA-TTS: Adapting Autoregressive Model into a Diffusion Language Model for Text-to-Speech (NeurIPS 2026 · similarity 0.85)
- NaturalSpeech 2: Latent Diffusion Models are Natural and Zero-Shot Speech and Singing Synthesizers (ICLR 2024 · similarity 0.85)
- DiTTo-TTS: Diffusion Transformers for Scalable Text-to-Speech without Domain-Specific Factors (ICLR 2025 · similarity 0.85)
- ELF: Encoding Speaker-Specific Latent Speech Feature for Speech Synthesis (ICML 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
