AI Paper Atlas

Research map / Label-efficient and robust learning / Self-supervised and contrastive

Better Correlation and Robustness: A Distribution-Balanced Self-Supervised Learning Framework for Automatic Dialogue Evaluation

Peiwen Yuan, Xinglin Wang, Jiayi Shi, Bin Sun, Yiwei Li, Kan Li

NeurIPS 2023 · Label-efficient and robust learning › Self-supervised and contrastive · approach: Data-efficient and robust self-supervision · 0 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
Remove whole training set reconstructing (TSR) (ablation)BERT+BCR · DSTC10 overallSpearman correlation: 0.261 -> 0.316Improved
Add pseudo label in TSR (ablation)BERT+BCR · DSTC10 overallSpearman correlation: 0.310 -> 0.316Improved
Add TSR on medium coherence samples (ablation)BERT+BCR · DSTC10 medium coherenceSpearman correlation: 0.073 -> 0.131Improved
Add TSR on polarized coherence samples (ablation)BERT+BCR · DSTC10 polarized coherenceSpearman correlation: 0.125 -> 0.163Improved
Apply BCR to baseline BERTBERT-base · 17 datasets averageSpearman correlation: 0.323 -> 0.415Improved
Apply BCR to BERT vs prior SOTA methodsBERT+BCR · 17 datasets averageSpearman correlation: 0.373 -> 0.415Improved
Apply BCR to BERT on GDRBERT+BCR · GDRSpearman correlation: 0.197 -> 0.487Improved
Apply BCR to BERT on PEBERT+BCR · PESpearman correlation: 0.612 -> 0.669Mixed
Apply BCR to GRADE and USRGRADE, USR · DSTC10Spearman correlation and Diff: 0.171 -> 0.242Improved
Use DP loss instead of MAE, CE, MSE or Triplet loss (ablation)BERT+BCR · DSTC10Spearman and robustness DiffImproved
Train on Empathetic instead of DailyDialogBERT+BCR · DSTC10Spearman correlation: 0.316 -> 0.191Hurt
BCR pre-training then supervised finetuning vs supervised onlyBERT · FED split 7:3Spearman correlation: 0.240 -> 0.372Improved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy