Research map / Large language models / Language, safety and interpretability
MBR and QE Finetuning: Training-time Distillation of the Best and Most Expensive Decoding Methods
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes MBR finetuning and QE finetuning, which distill MBR decoding and QE reranking gains into NMT models at training time so that efficient beam/greedy decoding can be used at inference, with self-teacher and PaLM-2 teacher variants.
Problem
MBR decoding and QE reranking improve NMT quality over beam search but are prohibitively expensive at inference time, since MBR is quadratic and reranking requires generating many candidates per input.
Approach
Generate MBR-decoded (BLEURT utility) or QE-reranked (MetricX-XXL-QE utility) translations from monolingual or reference-side sources, finetune the student on these top-1 outputs, and then decode the student efficiently with beam search at inference time; an external PaLM-2 Bison teacher can supply the candidates.
Key results
- On en-de WMT'22, PaLM-2-MBR-from-ref-finetuned reaches 63.62 COMET20 versus 57.79 for beam search base model and 60.95 for reference-finetuned, exceeding the WMT'22 top submission (63.26).
- Self-QE finetuning from the base model improves COMET20 from 57.79 to 59.63 (+1.84) on en-de, outperforming reference-finetuning on en-ja by 0.43 points (51.39 vs 50.96).
- Beam-search finetuning degrades en-de COMET20 from 57.79 to 57.08, while MBR and QE finetuning improve it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Finetune on MBR-decoded outputs instead of beam outputs | Base en→de Transformer · en→de WMT'22 test | COMET20: 57.79 -> 58.02; +0.23 | Helped |
| Finetune on QE-reranked outputs instead of beam outputs | Base en→de Transformer · en→de WMT'22 test | COMET20: 57.79 -> 59.63; +1.84 | Helped |
| Finetune on beam search translations | Base en→de Transformer · en→de WMT'22 test | COMET20: 57.79 -> 57.08; -0.71 | Hurt |
| Add self-QE finetuning on Common Crawl | Reference-finetuned en→de model · en→de WMT'22 test | COMET20: 60.95 -> 61.11; +0.16 | Helped |
| Add self-MBR finetuning on Common Crawl | Reference-finetuned en→de model · en→de WMT'22 test | COMET20: 60.95 -> 61.49; +0.54 | Helped |
| Use PaLM-2 rather than a self-teacher to generate QE data | Base en→de model · en→de WMT'22 test | COMET20: 60.49 -> 62.34; 1.85 points | Helped |
| Use PaLM-2 rather than a self-teacher to generate MBR data | Base en→de model · en→de WMT'22 test | COMET20: 59.94 -> 63.51; 3.57 points | Helped |
| Finetune on QE-reranked outputs rather than human references | Base en→ja Transformer · en→ja WMT'22 test | COMET20: 50.96 -> 51.39; 0.43 points | Helped |
| Use PaLM-2 rather than a self-teacher to generate QE data | Base en→ja model · en→ja WMT'22 test | COMET20: 53.73 -> 60.63 | Helped |
| Use greedy rather than beam decoding (ablation) | Phase 1 en→de QE-finetuned model · en→de WMT'22 test | COMET20: 59.63 -> 59.58 | No clear effect |
| Use sampling rather than beam decoding (ablation) | Phase 1 en→de QE-finetuned model · en→de WMT'22 test | COMET20: 59.63 -> 55.91 | Hurt |
| Reduce Common Crawl training data from 200k to 30k examples (ablation) | Reference-initialized MBR/QE finetuning · en→de WMT'22 test | COMET20: 61.49 -> 60.06; 61.11 -> 60.62 | Hurt |
Surprising findings
- Finetuning on beam search outputs degrades performance, while finetuning on MBR or QE outputs improves it.
- Self-teacher MBR and QE finetuning gains are small relative to using an external PaLM-2 teacher, and the paper argues the gains are not mainly explained by exposure-bias mitigation.
Limitations
- Self-teacher MBR and QE finetuning gains are modest compared to using an external LLM teacher.
- Metrics used for MBR/QE data generation (BLEURT, MetricX-XXL-QE) may bias evaluation, so Comet20 is the primary metric and MetricX scores are not fully unbiased.
- Experiments are limited to NMT with a small Transformer student and two or three language pairs.
Experimental setup
Datasets: WMT'21 en-de parallel training data WMT'21 NewsCrawl (backtranslation) WMT 2009-2019 newstest sets WMT'22 en-ja parallel training data WMT'20 en-ja test set Common Crawl (200k English sentences) WMT'22 generalMT test set newstest2021
Benchmarks: WMT'22 en-de generalMT test set WMT'22 en-ja test set WMT'22 de-en test set
Metrics: COMET20 BLEURT MetricX-XXL MetricX-23-c chrF COMET22 MQM score
Baselines: Beam search decoding QE reranking MBR decoding Reference-finetuned Beam-finetuned WMT 2022 top submission
Builds on: Minimum Bayes Risk decoding BLEURT MetricX ReST Minimum Bayes Risk Training
Closest papers by meaning
- Calibrating Translation Decoding with Quality Estimation on LLMs (NeurIPS 2025 · similarity 0.89)
- A Paradigm Shift in Machine Translation: Boosting Translation Performance of Large Language Models (ICLR 2024 · similarity 0.89)
- Decoding-Time Language Model Alignment with Multiple Objectives (NeurIPS 2024 · similarity 0.88)
- Model-Based Minimum Bayes Risk Decoding for Text Generation (ICML 2024 · similarity 0.87)
- QUEST: Quality-Aware Metropolis-Hastings Sampling for Machine Translation (NeurIPS 2024 · similarity 0.87)
- Uncertainty-Aware Decoding with Minimum Bayes Risk (ICLR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.