AI Paper Atlas

Research map / Label-efficient and robust learning / Recognition and long-tail

DirectQE: Direct Pretraining for Machine Translation Quality Estimation

Qu Cui, Shujian Huang, Jiahuan Li, Xiang Geng, Zaixiang Zheng, Guoping Huang, Jiajun Chen

AAAI 2021 · Label-efficient and robust learning › Recognition and long-tail · approach: Multilingual language representation pretraining · 20 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
Use DirectQE-Top-k instead of PLM-based QEQE on WMT19 · WMT19 testPearson: 52.08 -> 55.08Improved
Use DirectQE-Top-k instead of PLM-based QEQE on WMT19 · WMT19 testF1-MULT: 37.04 -> 39.71Improved
Use DirectQE-Top-k instead of PLM-based QEQE on WMT17 · WMT17 testPearson: 72.01 -> 73.56Improved
Use DirectQE ensemble instead of QE-Brain ensembleSentence-level QE · WMT17 testPearson: 71.59 -> 76.29Improved
Train the NMT-based QE system on generated pseudo pairs rather than parallel pairs (ablation)NMT-based QE · WMT19 testPearson: 50.65 -> 51.50Improved
Train the NMT-based QE system on generated pseudo pairs rather than parallel pairs (ablation)NMT-based QE · WMT17 testPearson: 67.44 -> 68.19Improved
Remove the sentence-level pretraining objective (ablation)DirectQE-Top-k · WMT19 testsentence-level QE: ($-$4.05)Hurt
Remove the sentence-level pretraining objective (ablation)DirectQE-Top-k · WMT19 testword-level QE: ($-$2.61)Hurt
Remove the sentence-level pretraining objective (ablation)DirectQE-Top-k · WMT17 testsentence-level QE: ($-$1.01)Hurt
Increase pseudo-translation replacement rate from low to moderate (ablation)DirectQE pseudo-data generation · Sentence-level QEPearsonImproved
Increase pseudo-translation replacement rate beyond the useful range (ablation)DirectQE pseudo-data generation · Sentence-level QEPearsonHurt
Use random replacement instead of quality-controlled sampling (ablation)DirectQE pseudo-data generation · Sentence-level QEPearsonHurt

Later papers that compared with this method

1 later accepted paper reports DirectQE as a baseline in their results tables; 1 of them report a better number on most of the results they share with it.

Matched by method name in results tables read by AI. Authors' own tables tend to favour the newer method, and a comparison can be missed when a method is renamed.

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy