Research map / Large language models / Alignment and preferences
SimulPL: Aligning Human Preferences in Simultaneous Machine Translation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SimulPL, a preference learning framework for simultaneous machine translation that builds GPT-4/4o-generated human-preferred references, trains with multi-task SFT (translation + read/write confidence), and then optimizes a latency-aware DPO variant (SimulDPO) on Llama2-7B-chat-based LLM SiMT models.
Problem
Simultaneous machine translation models are trained and evaluated on offline translation data and do not align with audience preferences (accuracy, monotonicity, concise key points, simple syntax, low latency); existing preference optimization methods ignore latency and the read/write policy.
Approach
Categorize five SiMT human preferences, use preference prompts to have GPT-4/4o produce human-preferred targets, jointly learn translation and a confidence-based read/write policy via MSFT, then apply SimulDPO which adds a latency term (alpha times expected output length) to the DPO reward so the policy learns when to write.
Key results
- On Zh-En low latency (LAAL<4), SimulPL reaches BLEU 23.73 vs 12.05 for LLM-PFX-SFT and 14.79 for LLM-PFX-SFT+DPO.
- On De-En low latency, SimulPL reaches COMET 82.34 and BLEU 42.69 versus 81.21 / 29.61 for LLM-PFX-SFT.
- Human evaluation by interpreters reports SimulPL with higher win rates than baselines across latency groups for all three language pairs (exact win rates in figure only).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use SimulPL instead of LLM-PFX-SFT+DPO | Llama2-7B-chat SiMT · Zh→En low latency | BLEU: 14.79 -> 23.73 | Helped |
| Use SimulPL instead of LLM-PFX-SFT+DPO | Llama2-7B-chat SiMT · De→En low latency | BLEU: 29.29 -> 42.69 | Helped |
| Use SimulPL instead of LLM-PFX-SFT+DPO | Llama2-7B-chat SiMT · En→Zh low latency | BLEU: 19.62 -> 38.99 | Helped |
| Use SimulPL instead of LLM-PFX-SFT + DPO | Llama2-7B-chat SiMT · Zh→En, n=3 | BLEU: 12.08 -> 22.51 | Helped |
| Use SimulPL instead of LLM-PFX-SFT + DPO | Llama2-7B-chat SiMT · Zh→En, n=3 | LAAL: 3.39 -> 1.73 | Helped |
| Add multi-task supervised fine-tuning (MSFT) rather than regular SFT (ablation) | SimulPL · Zh→En SiMT, especially low latency | SacreBLEU | Helped |
| Add SimulDPO after the MSFT phase (ablation) | SimulPL · Zh→En SiMT across latency levels | SacreBLEU | Helped |
| Replace the full-source reference probability π_ref(y|X) with the prefix-source probability π_ref(y|x) (ablation) | SimulPL · Zh→En SiMT | performance | Hurt |
| Set α to 0 instead of the default value (ablation) | SimulPL · Zh→En SiMT | performance | Hurt |
| Set α to 10 (ablation) | SimulPL · Zh→En SiMT | read/write policy | Hurt |
| Lower the inference confidence threshold to γ=0.1 (ablation) | SimulPL · Zh→En, low latency (0 ≤ LAAL < 4) | translation quality | Hurt |
| Raise the inference confidence threshold to γ=0.9 (ablation) | SimulPL · Zh→En SiMT | latency | Hurt |
Surprising findings
- Manually revised references scored lower than GPT-4/4o drafts on some metrics in the appendix, and GPT-4/4o translations were judged close to manually revised translations.
Limitations
- Preference evaluation relies on a small number of interpreters (100 sentences per group) and the test references are GPT-drafted and manually revised.
- Effect of alpha and confidence threshold gamma are only partially explored; authors defer further analysis to future work.
- Evaluation limited to three language pairs and text-to-text SiMT.
Experimental setup
Datasets: WMT15 De-En WMT22 Zh-En MUST-C En-Zh newstest2015 De-En newstest2021 Zh-En tst-COMMON
Benchmarks: newstest2015 De-En (manually revised references) newstest2021 Zh-En (manually revised references) tst-COMMON En-Zh (manually revised references)
Metrics: SacreBLEU COMET LAAL AL AP DAL NIR SLR Dependency Depth human win rate
Baselines: wait-k SM2 LLM-PFX-SFT LLM-PFX-SFT+DPO CPO KTO
Builds on: DPO CPO KTO R-DPO awesome-align
Closest papers by meaning
- SimPER: A Minimalist Approach to Preference Alignment without Hyperparameters (ICLR 2025 · similarity 0.89)
- Meta-Learning Preferences for Multilingual LLM Alignment (NeurIPS 2026 · similarity 0.88)
- Contrastive Preference Optimization: Pushing the Boundaries of LLM Performance in Machine Translation (ICML 2024 · similarity 0.88)
- Test-Time Preference Optimization: On-the-Fly Alignment via Iterative Textual Feedback (ICML 2025 · similarity 0.87)
- Preference Ranking Optimization for Human Alignment (AAAI 2024 · similarity 0.87)
- Spread Preference Annotation: Direct Preference Judgment for Efficient LLM Alignment (ICLR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.