Research map / Label-efficient and robust learning / Recognition and long-tail
Pre-training for Speech Translation: CTC Meets Optimal Transport
Phuong-Hang Le, Hongyu Gong, Changhan Wang, Juan Pino, Benjamin Lecouteux, Didier Schwab
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace CE with CTC for ASR pre-training | Vanilla ST models · MuST-C dev En→De and En→Fr | BLEU: 23.04 -> 24.08; 28.89 -> 29.91 | Improved |
| Add OT speech-text alignment to CTC pre-training | Siamese-PT · MuST-C dev En→De and En→Fr | BLEU: 24.08 -> 24.74; 29.91 -> 30.31 | Improved |
| Replace CE pre-training with CTC+OT Siamese-PT | Vanilla multilingual ST, large model · MuST-C test | BLEU: 28.8 -> 29.8 | Improved |
| Replace CTC with CTC+OT Siamese-PT | Vanilla multilingual ST · CoVoST-2 test En→X | BLEU: 19.8 -> 21.5 | Improved |
| Add CE alongside CTC during ASR pre-training (ablation) | Vanilla multilingual ST · MuST-C dev, multilingual average, with MT | BLEU: 26.52 -> 26.57 | No clear effect |
| Replace Wasserstein OT with Euclidean alignment and average pooling (ablation) | Siamese pre-training · MuST-C dev En→De | BLEU: 24.08 -> 24.41 | Improved |
| Replace Wasserstein OT with Euclidean alignment and attention matching (ablation) | Siamese pre-training · MuST-C dev En→Fr | BLEU: 29.91 -> 29.81 | No clear effect |
| Replace Wasserstein OT with adversarial alignment (ablation) | Siamese pre-training · MuST-C dev En→De | BLEU: 24.08 -> 23.73 | Hurt |
| Use Wasserstein OT rather than alternative distance functions (ablation) | Siamese pre-training · MuST-C dev En→De and En→Fr | BLEU: 24.56 -> 24.74; 30.10 -> 30.31 | Improved |
| Add positional encoding to OT (ablation) | Siamese-PT with MT · MuST-C dev En→De and En→Fr | BLEU: 24.29 -> 24.74; 30.04 -> 30.31 | Improved |
| Share speech and text encoder weights (ablation) | Siamese-PT with MT · MuST-C dev En→De and En→Fr | BLEU: 24.74 -> 23.99; 30.31 -> 29.68 | Hurt |
| Add CTC+OT Siamese pre-training to the multi-task system | Multi-task learning system · MuST-C dev En-De | BLEU: 26.78 -> 27.20 | Improved |
Closest papers by meaning
- Align With Purpose: Optimize Desired Properties in CTC Models with a General Plug-and-Play Framework (ICLR 2024 · similarity 0.87)
- CR-CTC: Consistency regularization on CTC for improved speech recognition (ICLR 2025 · similarity 0.86)
- High-Fidelity Simultaneous Speech-To-Speech Translation (ICML 2025 · similarity 0.86)
- Scaling Speech-Text Pre-training with Synthetic Interleaved Data (ICLR 2025 · similarity 0.86)
- CLaM-TTS: Improving Neural Codec Language Model for Zero-Shot Text-to-Speech (ICLR 2024 · similarity 0.86)
- HALL-E: Hierarchical Neural Codec Language Model for Minute-Long Zero-Shot Text-to-Speech Synthesis (ICLR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
