AI Paper Atlas

Research map / Label-efficient and robust learning / Recognition and long-tail

Pre-training for Speech Translation: CTC Meets Optimal Transport

Phuong-Hang Le, Hongyu Gong, Changhan Wang, Juan Pino, Benjamin Lecouteux, Didier Schwab

ICML 2023 · Label-efficient and robust learning › Recognition and long-tail · approach: Multilingual language representation pretraining · 7 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
Replace CE with CTC for ASR pre-trainingVanilla ST models · MuST-C dev En→De and En→FrBLEU: 23.04 -> 24.08; 28.89 -> 29.91Improved
Add OT speech-text alignment to CTC pre-trainingSiamese-PT · MuST-C dev En→De and En→FrBLEU: 24.08 -> 24.74; 29.91 -> 30.31Improved
Replace CE pre-training with CTC+OT Siamese-PTVanilla multilingual ST, large model · MuST-C testBLEU: 28.8 -> 29.8Improved
Replace CTC with CTC+OT Siamese-PTVanilla multilingual ST · CoVoST-2 test En→XBLEU: 19.8 -> 21.5Improved
Add CE alongside CTC during ASR pre-training (ablation)Vanilla multilingual ST · MuST-C dev, multilingual average, with MTBLEU: 26.52 -> 26.57No clear effect
Replace Wasserstein OT with Euclidean alignment and average pooling (ablation)Siamese pre-training · MuST-C dev En→DeBLEU: 24.08 -> 24.41Improved
Replace Wasserstein OT with Euclidean alignment and attention matching (ablation)Siamese pre-training · MuST-C dev En→FrBLEU: 29.91 -> 29.81No clear effect
Replace Wasserstein OT with adversarial alignment (ablation)Siamese pre-training · MuST-C dev En→DeBLEU: 24.08 -> 23.73Hurt
Use Wasserstein OT rather than alternative distance functions (ablation)Siamese pre-training · MuST-C dev En→De and En→FrBLEU: 24.56 -> 24.74; 30.10 -> 30.31Improved
Add positional encoding to OT (ablation)Siamese-PT with MT · MuST-C dev En→De and En→FrBLEU: 24.29 -> 24.74; 30.04 -> 30.31Improved
Share speech and text encoder weights (ablation)Siamese-PT with MT · MuST-C dev En→De and En→FrBLEU: 24.74 -> 23.99; 30.31 -> 29.68Hurt
Add CTC+OT Siamese pre-training to the multi-task systemMulti-task learning system · MuST-C dev En-DeBLEU: 26.78 -> 27.20Improved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy