Research map / Probabilistic and scientific modeling / Molecules and proteins
Improving Sequence-to-Sequence Learning via Optimal Transport
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add sequence-level OT and source-copy OT losses | GNMT · Machine translation, VI-EN and EN-VI | BLEU: 20.7 -> 21.9; 23.8 -> 25.5; 23.8 -> 24.5; 26.1 -> 26.9 | Helped |
| Add sequence-level OT and source-copy OT losses | GNMT · Machine translation, DE-EN and EN-DE | BLEU: 29.0 -> 29.2; 29.9 -> 30.1; 24.3 -> 24.6; 26.5 -> 26.8 | Helped |
| Add sequence-level OT and source-copy OT losses | Seq2Seq · Gigaword | ROUGE-1: 33.4 -> 36.2 | Helped |
| Add sequence-level OT and source-copy OT losses | Seq2Seq · DUC-2004 | ROUGE-1: 28.0 -> 30.1 | Helped |
| Add sequence-level OT loss | Show, Attend & Tell · COCO, soft attention + FastRCNN | CIDEr: 99.1 -> 102.9 | Helped |
| Use Gumbel-softmax instead of soft-argmax for differentiable token beliefs (ablation) | GNMT+OT · VI-EN NT2012 and EN-VI NT2013 | BLEU: 20.7 -> 20.9; 25.4 -> 25.7 | Mixed |
| Use standard softmax instead of soft-argmax for differentiable token beliefs (ablation) | GNMT+OT · VI-EN NT2013 and EN-VI NT2013 | BLEU: 23.8 -> 24.3; 25.4 -> 26.0 | Mixed |
| Allow OT gradients to update the sequence model as well as word embeddings (ablation) | Seq2Seq autoencoder · WMT English reconstruction | BLEU: Baseline 71.87; Case 1 73.60; Case 2 75.12 | Helped |
| Use OT gradients only to refine the word embedding matrix, not the sequence model (ablation) | Seq2Seq autoencoder · WMT English reconstruction | BLEU: 71.87 -> 73.60 | Helped |
Closest papers by meaning
- MLE-Guided Parameter Search for Task Loss Minimization in Neural Sequence Modeling (AAAI 2021 · similarity 0.89)
- Sentence-Wise Smooth Regularization for Sequence to Sequence Learning (AAAI 2019 · similarity 0.89)
- Sequence-to-Sequence Learning with Latent Neural Grammars (NeurIPS 2021 · similarity 0.87)
- Minimum Divergence vs. Maximum Margin: an Empirical Comparison on Seq2Seq Models (ICLR 2019 · similarity 0.87)
- Multi-Level Optimal Transport for Universal Cross-Tokenizer Knowledge Distillation on Language Models (AAAI 2025 · similarity 0.85)
- SeqGPT: An Out-of-the-Box Large Language Model for Open Domain Sequence Understanding (AAAI 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.