Research map / Image generation / Autoregressive and latent generation
A Vector Quantized Approach for Text to Speech Synthesis on Real-World Spontaneous Speech
Li-Wei Chen, Shinji Watanabe, Alexander Rudnicky
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace one codebook with four codebooks (ablation) | MQTTS speech quantizer · VoxCeleb test set | RCER (%): 56.5 -> 19.7 | Improved |
| Increase one codebook from 1024 to 65536 codes (ablation) | MQTTS speech quantizer · VoxCeleb test set | RCER (%): 56.5 -> 59.9 | No clear effect |
| Increase codebook groups from four to eight (ablation) | MQTTS speech quantizer · VoxCeleb test set | RCER (%): 19.7 -> 14.2 | Improved |
| Scale model size from approximately 40M to approximately 200M parameters (ablation) | MQTTS · GigaSpeech real-world speech | WER (%): 24.9 -> 22.2 | Improved |
| Scale model size from approximately 40M to approximately 200M parameters (ablation) | MQTTS · GigaSpeech real-world speech | MOS-N: 3.79$\pm$0.06 -> 3.89$\pm$0.06 | Improved |
| Remove monotonic alignment (ablation) | MQTTS, approximately 100M · GigaSpeech real-world speech | WER (%): 22.3 -> 34.3 | Hurt |
| Add monotonic alignment (ablation) | MQTTS, approximately 100M · GigaSpeech real-world speech | MOS-N: 3.76$\pm$0.06 -> 3.87$\pm$0.06 | Improved |
| Replace the sub-decoder with the same number of linear layers (ablation) | MQTTS, approximately 100M · GigaSpeech real-world speech | P-FID: 8.58 -> 12.12 | Hurt |
| Add the sub-decoder for conditional code sampling (ablation) | MQTTS, approximately 100M · GigaSpeech real-world speech | MOS-N: 3.75$\pm$0.07 -> 3.87$\pm$0.06 | Improved |
| Replace attention alignment with a duration predictor (ablation) | MQTTS, approximately 100M · GigaSpeech real-world speech | WER (%): 22.3 -> 53.2 | Hurt |
| Use an autoregressive multi-code model instead of VITS | MQTTS and VITS, approximately 100M · GigaSpeech real-world speech | P-FID: 12.04 -> 8.58 | Improved |
| Use MQTTS instead of VITS | MQTTS and VITS, approximately 100M · GigaSpeech real-world speech | MCD: 8.73$\pm$1.19 -> 10.22$\pm$1.68 | Hurt |
Closest papers by meaning
- CLaM-TTS: Improving Neural Codec Language Model for Zero-Shot Text-to-Speech (ICLR 2024 · similarity 0.89)
- Scaling Speech-Text Pre-training with Synthetic Interleaved Data (ICLR 2025 · similarity 0.88)
- NaturalSpeech 2: Latent Diffusion Models are Natural and Zero-Shot Speech and Singing Synthesizers (ICLR 2024 · similarity 0.87)
- DELTA-TTS: Adapting Autoregressive Model into a Diffusion Language Model for Text-to-Speech (NeurIPS 2026 · similarity 0.87)
- ELLA-V: Stable Neural Codec Language Modeling with Alignment-Guided Sequence Reordering (AAAI 2025 · similarity 0.87)
- NaturalSpeech 3: Zero-Shot Speech Synthesis with Factorized Codec and Diffusion Models (ICML 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
