AI Paper Atlas

Research map / Large language models / Language, safety and interpretability

Masking Orchestration: Multi-Task Pretraining for Multi-Role Dialogue Representation Learning

Tianyi Wang, Yating Zhang, Xiaozhong Liu, Changlong Sun, Qiong Zhang

AAAI 2020 · Large language models › Language, safety and interpretability · approach: Sign and cross-lingual translation · 11 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
Pretrain with multi-task masking rather than random initializationOur model · CDD JFRmacro F1: 39.61 -> 45.45Improved
Pretrain with multi-task masking rather than random initializationOur model · CSD DARmacro F1: 76.83 -> 78.28Improved
Pretrain with multi-task masking rather than random initializationOur model · EMD DARmacro F1: 52.78 -> 53.69Improved
Pretrain with multi-task masking rather than random initializationOur model · CDD CFGBLEU-4: 2.94 -> 8.59Improved
Pretrain with multi-task masking rather than random initializationOur model · EMD DSBLEU-4: 14.87 -> 23.20Improved
Add word prediction during pretraining (ablation)Our model · CDD JFRmacro F1: 41.66 -> 45.45Improved
Add role prediction during pretraining (ablation)Our model · CDD JFRmacro F1: 40.78 -> 45.45Improved
Add sentence generation during pretraining (ablation)Our model · CDD JFRmacro F1: 43.81 -> 45.45Improved
Add reference prediction during pretraining (ablation)Our model · CDD JFRmacro F1: 41.71 -> 45.45Improved
Add word prediction during pretraining (ablation)Our model · CDD CFGBLEU-4: 8.09 -> 8.59Improved
Add role prediction during pretraining (ablation)Our model · CDD CFGBLEU-4: 8.22 -> 8.59Improved
Add sentence generation during pretraining (ablation)Our model · CDD CFGBLEU-4: 5.58 -> 8.59Improved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy