Research map / Label-efficient and robust learning / Recognition and long-tail
Learning from others' mistakes: Finetuning machine translation models with span-level error annotations
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Training with Annotations (TWA), a finetuning method that uses span-level MQM error annotations for machine translation via a span-level unlikelihood loss on error spans and cross-entropy on non-error prefix tokens.
Problem
Most feedback-based training of language models uses only sequence-level labels; span-level error annotations in offline MT datasets (MQM) have not been used to directly finetune MT models.
Approach
TWA penalizes error spans with a severity-weighted span-level unlikelihood loss (letting the model learn which tokens within the span to push down), trains on non-error tokens before the first error with cross entropy, and ignores off-trajectory tokens after the first error.
Key results
- On En-De with submissions and references, TWA reaches MetricX 2.882 and COMET 0.513, versus 3.159 / 0.491 for SFT with references and 3.564 / 0.442 for DPO.
- On Zh-En with submissions and references, TWA obtains MetricX 3.965 and COMET 0.290, the best in both columns, with clustering placing it in the top cluster.
- Replacing unlikelihood with negative likelihood on error spans worsens En-De MetricX from 2.944 to 3.477 and Zh-En MetricX from 4.091 to 4.730.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use TWA instead of SFT on submissions | 602M Transformer · WMT'23 En→De test | Metric-X: 3.573 -> 2.944 | Helped |
| Use TWA instead of SFT on submissions | 602M Transformer · WMT'23 Zh→En test | COMET: 0.255 -> 0.277 | Helped |
| Use TWA rather than Filter + SFT | 602M Transformer · WMT'23 En→De test | COMET: 0.499 -> 0.513 | Helped |
| Train on submissions instead of the base model alone (ablation) | 602M Transformer · WMT'23 En→De test | Metric-X: 4.203 -> 3.573 | Helped |
| Train only on non-error tokens rather than all submission tokens (ablation) | TWA · WMT'23 En→De test | Metric-X: 3.573 -> 3.488 | Helped |
| Apply span-level loss to annotated error spans instead of ignoring error tokens (ablation) | TWA · WMT'23 En→De test | Metric-X: 3.488 -> 3.325 | Helped |
| Ignore off-trajectory tokens after the first error (ablation) | TWA · WMT'23 En→De test | Metric-X: 3.325 -> 2.944 | Helped |
| Ignore off-trajectory tokens after the first error (ablation) | TWA · WMT'23 Zh→En test | COMET: 0.284 -> 0.277 | Hurt |
| Replace span-level unlikelihood with span-level negative likelihood (ablation) | TWA error-span loss · WMT'23 En→De test | Metric-X: 2.944 -> 3.477 | Hurt |
| Replace span-level unlikelihood with span-level negative likelihood (ablation) | TWA error-span loss · WMT'23 Zh→En test | COMET: 0.277 -> 0.108 | Hurt |
| Use DPO instead of TWA on submissions and references | 602M Transformer · WMT'23 En→De test | Metric-X: 3.564 -> 2.882 | Hurt |
| Use sequence-level unlikelihood instead of span-level TWA | 602M Transformer · WMT'23 En→De test | Metric-X: 3.158 -> 2.882 | Hurt |
Surprising findings
- Sequence-level baselines that use negative information (DPO, TWA-seq) do not reliably improve over SFT.
- Ignoring off-trajectory tokens gives substantial gains for En-De but not for Zh-En.
Limitations
- Evaluated only on En-De and Zh-En machine translation with MQM data.
- Off-trajectory token handling helps one language pair but not the other.
- MetricX-23 was finetuned on MQM WMT'20-'21 data, which may bias evaluation despite the use of COMET.
Experimental setup
Datasets: WMT'23 parallel training data WMT'20 MQM data WMT'21 MQM data generalMT2022 test set WMT'23 test set
Benchmarks: WMT'23 En-De test set WMT'23 Zh-En test set
Metrics: MetricX-23 COMET-20
Baselines: SFT DPO Filter + SFT TWA-seq Base model
Builds on: Direct Preference Optimization Targeted Negative Training FG-RLHF
Closest papers by meaning
- A Paradigm Shift in Machine Translation: Boosting Translation Performance of Large Language Models (ICLR 2024 · similarity 0.88)
- When Scaling Meets LLM Finetuning: The Effect of Data, Model and Finetuning Method (ICLR 2024 · similarity 0.87)
- Error Norm Truncation: Robust Training in the Presence of Data Noise for Text Generation Models (ICLR 2024 · similarity 0.86)
- MT-Ranker: Reference-free machine translation evaluation by inter-system ranking (ICLR 2024 · similarity 0.86)
- Beyond Next Token Prediction: Patch-Level Training for Large Language Models (ICLR 2025 · similarity 0.86)
- Teaching Large Language Models to Translate with Comparison (AAAI 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.