AI Paper Atlas

Research map / Large language models / Language, safety and interpretability

Multitasking Models are Robust to Structural Failure: A Neural Model for Bilingual Cognitive Reserve

Giannis Daras, Negin Raoof, Zoi Gkalitsiou, Alex Dimakis

NeurIPS 2022 · Large language models › Language, safety and interpretability · approach: Reasoning mechanisms and failure modes · 1 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
train bilingual (English+Greek) instead of monolingual GPT-2GPT-2 small · GLUE SST2 under random pruning 0.45accuracy: 0.651 -> 0.773Improved
train bilingual instead of monolingual GPT-2GPT-2 small · GLUE MRPC under random pruning 0.20accuracy: 0.416 -> 0.663Improved
train bilingual instead of monolingual GPT-2GPT-2 small · GLUE CoLA under random pruning 0.30MCC: 0.216 -> 0.137Hurt
train bilingual instead of monolingual GPT-2GPT-2 small · GLUE RTE under random pruning 0.30accuracy: 0.537 -> 0.536No clear effect
train bilingual instead of monolingual GPT-2GPT-2 small · GLUE QQP, no perturbationaccuracy: 0.876 -> 0.843Hurt
train bilingual instead of monolingual GPT-2GPT-2 small · GLUE MRPC under Gaussian noise std 0.06accuracy: 0.316 -> 0.610Improved
train bilingual instead of monolingual GPT-2GPT-2 small · GLUE CoLA under Gaussian noise std 0.04MCC: 0.316 -> 0.146Hurt
train bilingual instead of monolingual GPT-2GPT-2 small · GLUE SST2 under Gaussian noise std 0.07accuracy: 0.545 -> 0.568Improved
fine-tune from bilingual checkpoint instead of monolingualGPT-2 small · GLUE SST2 attention weightssum of squared weights: 70567.875 -> 60663.121Improved
train bilingual instead of monolingual GPT-2GPT-2 small · IMDB perplexity, no perturbationperplexity: 36 -> 76Hurt
delete increasing share of attention weights, bilingual vs monolingualGPT-2 small · IMDB perplexity under random deletionperplexityImproved
train bilingual for more epochs (1 vs 8)bilingual GPT-2 · IMDB perplexity under magnitude pruningperplexityImproved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy