Research map / Image generation / Autoregressive and latent generation
Translatotron 2: High-quality direct speech-to-speech translation with voice preservation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace the original Translatotron architecture with Translatotron 2's linguistic-decoder and acoustic-synthesizer architecture | Direct S2ST · Fisher Es-En test | BLEU: 26.9 -> 42.4 (+15.5 BLEU) | Helped |
| Replace the original Translatotron architecture with Translatotron 2's linguistic-decoder and acoustic-synthesizer architecture | Direct S2ST · Conversational test | BLEU: 50.4 -> 55.6 (+5.2 BLEU) | Helped |
| Replace the original Translatotron architecture with Translatotron 2's linguistic-decoder and acoustic-synthesizer architecture | Direct S2ST · Conversational test | UDR (%): 0.69 -> 0.16 | Helped |
| Add ConcatAug | Translatotron 2 · Fisher Es-En test | BLEU: 42.4 -> 42.9 (+0.5 BLEU) | Helped |
| Add ConcatAug | Translatotron 2 · Conversational test | BLEU: 55.6 -> 55.1 | Hurt |
| Add ConcatAug | Translatotron 2 · Conversational test | Naturalness MOS: 4.21 ± 0.06 -> 4.19 ± 0.06 | Hurt |
| Add ConcatAug training examples containing concatenated speakers | Translatotron 2 · Speaker turns | Speaker similarity MOS on matched speaker segments: 2.22 ± 0.07 -> 2.44 ± 0.07; 2.00 ± 0.07 -> 2.51 ± 0.08 | Helped |
| Add ConcatAug (ablation) | Translatotron 2 · CoVoST 2 fr→en | BLEU: 27.0 -> 25.9 | Helped |
| Remove SpecAugment (ablation) | Translatotron 2 · CoVoST 2 fr→en | BLEU: 27.0 -> 25.9 | Hurt |
| Remove SpecAugment (ablation) | Translatotron 2 · CoVoST 2 de→en | BLEU: 18.8 -> 17.9 | Hurt |
| Replace the Conformer encoder with a bidirectional LSTM encoder (ablation) | Translatotron 2 · CoVoST 2 fr→en | BLEU: 27.0 -> 26.4 | Hurt |
| Replace the NAT synthesizer with a non-autoregressive Conformer synthesizer (ablation) | Translatotron 2 · CoVoST 2 fr→en | BLEU: 27.0 -> 26.9 | Hurt |
Closest papers by meaning
- TransVIP: Speech to Speech Translation System with Voice and Isochrony Preservation (NeurIPS 2024 · similarity 0.90)
- RVCBench: Benchmarking Robustness of Voice Cloning Across Modern Audio Generation Models (NeurIPS 2026 · similarity 0.85)
- High-Fidelity Simultaneous Speech-To-Speech Translation (ICML 2025 · similarity 0.85)
- SelfVC: Voice Conversion With Iterative Refinement using Self Transformations (ICML 2024 · similarity 0.85)
- NaturalSpeech 2: Latent Diffusion Models are Natural and Zero-Shot Speech and Singing Synthesizers (ICLR 2024 · similarity 0.85)
- E2E-VGuard: Adversarial Prevention for Production LLM-based End-To-End Speech Synthesis (NeurIPS 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.