Research map / Reinforcement learning / Offline RL
Sequence Generation with Optimal-Transport-Enhanced Reinforcement Learning
Liqun Chen, Ke Bai, Chenyang Tao, Yizhe Zhang, Guoyin Wang, Wenlin Wang, Ricardo Henao, Lawrence Carin
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add OT loss and RL annealing (OTRL) to MLE | Seq2seq LSTM · IWSLT14 EN-DE | BLEU: 26.44 -> 28.13 | Improved |
| OTRL instead of ERPO (RL only) | Seq2seq LSTM · IWSLT14 EN-DE | BLEU: 27.82 -> 28.13 | Improved |
| OTRL instead of Seq2Seq-OT (OT only) | Seq2seq LSTM · Gigaword | ROUGE-1: 36.82 -> 37.2 | Improved |
| add OT loss and RL to MLE | Seq2seq LSTM · Gigaword | ROUGE-1: 36.11 -> 37.2 | Improved |
| add RL (self-critic) on top of OT loss (ablation) | Pointer-Generator + Coverage Penalty + OT · CNN/Daily-mail | ROUGE-1: 40.28 -> 41.40 | Improved |
| add OT loss to pointer-generator with coverage penalty (ablation) | Pointer-Generator + Coverage Penalty · CNN/Daily-mail | ROUGE-1: 39.12 -> 40.28 | Improved |
| add OT loss to MLE (ablation) | LSTM captioner · COCO Captions | CIDEr: 106.3 -> 107.9 | Improved |
| add OT loss to self-critic RL (OTRL vs SC) (ablation) | LSTM captioner · COCO Captions | CIDEr: 110.4 -> 111.8 | Improved |
| add OT loss to self-critic RL (OTRL vs SC) (ablation) | LSTM captioner · COCO Captions | BLEU-1: 79.5 -> 79.3 | Mixed |
| add OT loss regularizer when switching MLE to RL (ablation) | LSTM captioner · COCO val | CIDEr stability | Improved |
| add OT only (MLE+OT) without RL (ablation) | Seq2seq LSTM · Gigaword | ROUGE-1 convergence | No clear effect |
Closest papers by meaning
- ESRL: Efficient Sampling-Based Reinforcement Learning for Sequence Generation (AAAI 2024 · similarity 0.89)
- Sequence to Sequence Reward Modeling: Improving RLHF by Language Feedback (AAAI 2025 · similarity 0.89)
- Towards Cost-Effective Reward Guided Text Generation (ICML 2025 · similarity 0.88)
- Degeneration-free Policy Optimization: RL Fine-Tuning for Language Models without Degeneration (ICML 2024 · similarity 0.88)
- Reducing the Probability of Undesirable Outputs in Language Models Using Probabilistic Inference (NeurIPS 2025 · similarity 0.87)
- MA-RLHF: Reinforcement Learning from Human Feedback with Macro Actions (ICLR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
