Research map / Image generation / Autoregressive and latent generation
TTS Can Speak in Any Style with Any Voice
Dekun Chen, Xueyao Zhang, Yuancheng Wang, Kenan Dai, Li Ma, Zhizheng Wu
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add S1 multi-modality DPO before S3 (vs S3 only) (ablation) | FlexiVoice PPT · InstructTTSEval-EN | Avg. accuracy: 72.3 -> 79.3 | Improved |
| run S1 then S2 then S3 rather than S3 -> S1 -> S2 (ablation) | FlexiVoice PPT · InstructTTSEval-EN | Avg. accuracy: 74.8 -> 79.3 | Improved |
| train S2 and S3 progressively instead of jointly (ablation) | FlexiVoice PPT · Emotion decoupling set (EN) | Decoupling Avg.: 84.1 -> 88.7 | Improved |
| train S2 and S3 progressively instead of jointly (ablation) | FlexiVoice PPT · InstructTTSEval-EN | Avg. accuracy: 75.5 -> 79.3 | Improved |
| add S1 multi-modality DPO to base model (ablation) | FlexiVoice-Base · Emotion decoupling set (EN) | Decoupling Avg.: +28.3 | Improved |
| add S2 decoupling GRPO after S1 (ablation) | FlexiVoice after S1 · Emotion decoupling set (EN) | Decoupling Avg.: 83.2 -> 88.5 | Improved |
| add S3 instruction GRPO after S1 and S2 (ablation) | FlexiVoice after S1+S2 · InstructTTSEval-EN | Avg. accuracy: 71.7 -> 79.3 (+7.6) | Improved |
| add S3 instruction GRPO after S1 and S2 (ablation) | FlexiVoice after S1+S2 · Emotion decoupling set (EN) | Decoupling Avg.: 88.5 -> 88.7 | No clear effect |
| use speaker verification reward instead of speaker similarity reward in decoupling GRPO (ablation) | FlexiVoice S2 · Emotion decoupling set TR-easy/TR-hard (EN) | ACC-I | Improved |
| apply full progressive post-training to pretrained model | FlexiVoice-Base · InstructTTSEval-EN | Avg. accuracy: 66.4 -> 79.3 | Improved |
| apply full progressive post-training to pretrained model | FlexiVoice-Base · Emotion TR-hard (EN) | ACC-I: 48.8 -> 78.2 | Improved |
| apply progressive post-training to pretrained model | FlexiVoice-Base · Emotion TO-easy (EN) | WER: 5.01 -> 5.99 | Hurt |
Closest papers by meaning
- Mega-TTS 2: Boosting Prompting Mechanisms for Zero-Shot Speech Synthesis (ICLR 2024 · similarity 0.87)
- Vevo: Controllable Zero-Shot Voice Imitation with Self-Supervised Disentanglement (ICLR 2025 · similarity 0.86)
- MM-TTS: Multi-Modal Prompt Based Style Transfer for Expressive Text-to-Speech Synthesis (AAAI 2024 · similarity 0.86)
- StableVC: Style Controllable Zero-Shot Voice Conversion with Conditional Flow Matching (AAAI 2025 · similarity 0.86)
- ELLA-V: Stable Neural Codec Language Modeling with Alignment-Guided Sequence Reordering (AAAI 2025 · similarity 0.84)
- Do Not Mimic My Voice : Speaker Identity Unlearning for Zero-Shot Text-to-Speech (ICML 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
