Research map / Architectures and efficiency / Attention and transformers
Neural Speech Synthesis with Transformer Network
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace Tacotron2's recurrent encoder and decoder with Transformer self-attention | Tacotron2 · Internal US English female evaluation set (38 sentences) | CMOS: 0 -> 0.048 | Helped |
| replace recurrent encoder and decoder with Transformer layers | Tacotron2 · Internal US English female dataset | training time per step: ~1.7s -> ~0.4s; 4.25 times faster | Helped |
| add pre-net output re-centering (ablation) | Transformer TTS · Internal US English female dataset | MOS: 4.32 ± 0.05 -> 4.36 ± 0.05 | Helped |
| use trainable-scale rather than original fixed-scale positional encoding (ablation) | Transformer TTS · Internal US English female dataset | MOS: 4.37 ± 0.05 -> 4.40 ± 0.05 | Helped |
| use trainable absolute position embeddings (ablation) | Transformer TTS · long sequences | phoneme preservation | Hurt |
| increase encoder and decoder depth from 3 to 6 layers (ablation) | Transformer TTS · Internal US English female dataset | MOS: 4.33 ± 0.06 -> 4.41 ± 0.05 | Helped |
| increase attention heads from 4 to 8 (ablation) | Transformer TTS · Internal US English female dataset | MOS: 4.39 ± 0.05 -> 4.44 ± 0.05 | Helped |
| reduce encoder and decoder depth from 6 to 3 layers (ablation) | Transformer TTS · Internal US English female dataset | training time per step: $0.50$~~~ -> $0.29$ | Helped |
| reduce attention heads from 8 to 4 (ablation) | Transformer TTS · Internal US English female dataset | training time per step: $0.50$~~~ -> $0.44$~~~ | Helped |
| increase decoder pre-net hidden size from 256 to 512 (ablation) | Transformer TTS · Internal US English female dataset | model performance | No clear effect |
| remove nonlinear activation from the two-layer decoder pre-net (ablation) | Transformer TTS · encoder-decoder attention alignment | alignment quality | Hurt |
| make dot-product multi-head attention location-sensitive (ablation) | Transformer TTS · training | training time: doubles the training time | Hurt |
Closest papers by meaning
- HALL-E: Hierarchical Neural Codec Language Model for Minute-Long Zero-Shot Text-to-Speech Synthesis (ICLR 2025 · similarity 0.85)
- Aligner-Encoders: Self-Attention Transformers Can Be Self-Transducers (NeurIPS 2024 · similarity 0.85)
- Mega-TTS 2: Boosting Prompting Mechanisms for Zero-Shot Speech Synthesis (ICLR 2024 · similarity 0.84)
- CLaM-TTS: Improving Neural Codec Language Model for Zero-Shot Text-to-Speech (ICLR 2024 · similarity 0.84)
- DiTTo-TTS: Diffusion Transformers for Scalable Text-to-Speech without Domain-Specific Factors (ICLR 2025 · similarity 0.84)
- TTFSFormer: A TTFS-based Lossless Conversion of Spiking Transformer (ICML 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.