AI Paper Atlas

Research map / Large language models / Language, safety and interpretability

In-Context Impersonation Reveals Large Language Models' Strengths and Biases

Leonard Salewski, Stephan Alaniz, Isabel Rio-Torto, Eric Schulz, Zeynep Akata

NeurIPS 2023 · Large language models › Language, safety and interpretability · approach: Faithfulness of LLM explanations · 36 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
Increase the impersonated age from 2 to 20Vicuna-13B and Llama 2 · Two-armed bandit, ages 2–20average reward: β=0.17 for Vicuna-13B; β=0.26 for Llama 2Improved
Increase the impersonated age from 2 to 20Vicuna-13B · Two-armed bandit, ages 2–20directed exploration: β=-0.03Hurt
Increase the impersonated age from 2 to 20Vicuna-13B · Two-armed bandit, ages 2–20exploitation: β=0.04Improved
Increase the impersonated age from 20 to 60Vicuna-13B · Two-armed banditaverage rewardNo clear effect
Impersonate a task expert instead of a non-domain expertVicuna-13B · MMLUaccuracyImproved
Impersonate a domain expert instead of a non-domain expertVicuna-13B and ChatGPT · MMLUaccuracyImproved
Use the official MMLU prompt instead of the paper's task formulation (ablation)Vicuna-13B and ChatGPT with expert personas · MMLUaccuracyImproved
Add a social-category prefix to expert personas (ablation)Vicuna-13B and ChatGPT · MMLUaccuracyHurt
Impersonate an older person rather than a 2-year-oldChatGPT · Stanford Carsclassification accuracy: ∼4% to 54.9%Improved
Impersonate the topic expert rather than a non-expertChatGPT · CUB and Stanford Carsclassification accuracyImproved
Impersonate a man rather than a womanVicuna-13B · Stanford Carsclassification accuracyImproved
Impersonate a woman rather than a manVicuna-13B · CUBclassification accuracyImproved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy