AI Paper Atlas

Research map / Large language models / Retrieval and knowledge

UNICORN on RAINBOW: A Universal Commonsense Reasoning Model on a New Multitask Benchmark

Nicholas Lourie, Ronan Le Bras, Chandra Bhagavatula, Yejin Choi

AAAI 2021 · Large language models › Retrieval and knowledge · approach: Explanatory and multiple-choice QA benchmarks · 78 citations (OpenAlex)

Read the paper

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
Train on the other Rainbow tasks, then fine-tune on the targetT5-Large · Rainbow tasksdev accuracy: ANLI 77.8 -> 79.5; CosmosQA 81.9 -> 83.2; HellaSwag 82.8 -> 83.0; PhysicalIQA 80.2 -> 82.2; SocialIQA 73.8 -> 75.5; WinoImproved
Train on multiple tasks simultaneously, including the targetT5-Large · Rainbow tasksdev accuracy: WinoGrande 77.0 -> 72.1Mixed
Use Rainbow rather than GLUE or SuperGLUE for sequential transferT5-Large · Rainbow tasksdata efficiencyImproved
Add commonsense knowledge-graph generation as a multitask sourceT5-Large · Rainbow tasksdev accuracy: ATOMIC: ANLI 77.8 -> 78.3; CosmosQA 81.9 -> 81.8; HellaSwag 82.8 -> 82.8; PhysicalIQA 80.2 -> 79.9; SocialIQA 73.8 -> 75No clear effect
Scale the pretrained model up when using transferT5 models transferred from Rainbow to CommonsenseQA · CommonsenseQAdata efficiencyImproved
Use sequential training instead of multitask training or multitask fine-tuningT5 models · Rainbow tasksdata efficiencyImproved
Initialize from T5-11B multitasked on Rainbow, then fine-tune on each target taskT5-11B · ANLIaccuracy: 86.8% -> 87.3%Improved
Initialize from T5-11B multitasked on Rainbow, then fine-tune on each target taskT5-11B · CosmosQAaccuracy: 90.3% -> 91.8%Improved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy