Research map / Large language models / Language, safety and interpretability
Bias Runs Deep: Implicit Reasoning Biases in Persona-Assigned LLMs
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Shows that assigning socio-demographic personas to LLMs degrades their performance on 24 reasoning datasets, often via stereotyped abstentions, and that simple prompt-based de-biasing barely helps.
Problem
Persona assignment is used to personalize LLMs and simulate human behavior, but its unintended effects on basic reasoning ability and on socio-demographic bias are unknown.
Approach
Systematically assign 19 socio-demographic personas via system-prompt persona instructions and measure accuracy drops across 24 MMLU/Sports/MBPP datasets, separating explicit abstentions from implicit errors, and testing prompt-based de-biasing.
Key results
- ChatGPT (gpt-3.5-turbo-0613) shows persona-induced bias in 80% of 19 personas; Phys. Disabled persona averages 33-35% relative drop in accuracy.
- Relative drops reach 69% on a single dataset (Religious persona, college chemistry) and 64% (Phys. Disabled, high school world history).
- GPT-4-Turbo shows the least bias, affecting 42% of personas; de-biasing prompts such as 'don't refuse' or 'no stereotypes' have minimal effect, while dataset-specific expert personas reduce bias but do not generalize.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Assign a socio-demographic persona instead of the Human baseline | gpt-3.5-turbo-0613 · 24 reasoning datasets | accuracy: 35%+ | Hurt |
| Assign a physically-disabled persona instead of an able-bodied persona | gpt-3.5-turbo-0613 · 24 reasoning datasets | accuracy: 33% | Hurt |
| Assign a Religious persona instead of a Jewish persona | gpt-3.5-turbo-0613 · 24 reasoning datasets | accuracy: 28% | Hurt |
| Assign a Religious persona instead of the Human baseline | gpt-3.5-turbo-0613 · college chemistry | accuracy: 69% | Hurt |
| Assign a physically-disabled persona instead of the Human baseline | gpt-3.5-turbo-0613 · high school world history | accuracy: 64% | Hurt |
| Assign a Religious persona instead of an Atheist persona | gpt-3.5-turbo-0613 · college physics | accuracy: 56% | Hurt |
| Assign a Trump Supporter persona instead of an Obama Supporter persona | gpt-3.5-turbo-0613 · moral scenarios | accuracy: 27% | Hurt |
| Append task-agnostic debiasing instructions such as don't refuse, no stereotypes, or treat human (ablation) | gpt-3.5-turbo-0613 · reasoning datasets | relative accuracy drop between personas | No clear effect |
| Add task-specific expertise to the persona (ablation) | gpt-3.5-turbo-0613 · four MMLU datasets | relative accuracy drop between Able-Bodied and Phys. Disabled personas: 70.6 -> 2.5; 41.7 -> 0.5; 37.5 -> 9.2; 26.1 -> 12.0 | Helped |
| Append Treat Human #2 instruction (ablation) | gpt-3.5-turbo-0613 · college physics, Atheist vs Religious | relative % accuracy drop: No Mitigation: 26.1; Treat Human #2: -4.2 | Hurt |
| Use GPT-4-Turbo instead of gpt-3.5-turbo-0613 | persona-assigned LLMs · 24 reasoning datasets | personas with statistically significant performance difference from Human: 5 out of 12 | Helped |
| Use gpt-3.5-turbo-1106 instead of gpt-3.5-turbo-0613 | persona-assigned LLMs · 24 reasoning datasets | personas with statistically significant drops from Human: all 12 personas | Hurt |
Surprising findings
- Models explicitly deny stereotypes when asked directly yet abstain citing persona-based limitations when role-playing.
- Treat-human de-biasing instruction increased bias relative to no mitigation for the Atheist vs Religious pair.
Limitations
- Persona set is not exhaustive and skews toward majority and WEIRD groups.
- Only English datasets and prompts were used.
- Results for the non-primary models use a single run and a single persona instruction.
Experimental setup
Datasets: MMLU (22 subsets) Sports Understanding (BIG-Bench Hard) MBPP
Benchmarks: MMLU Sports Understanding MBPP
Metrics: accuracy relative accuracy drop abstention rate
Baselines: No Persona Human persona Average Human persona
Builds on: MMLU Big-Bench-Hard MBPP Wilson confidence interval
Closest papers by meaning
- LLM Generated Persona is a Promise with a Catch (NeurIPS 2025 · similarity 0.88)
- From Single to Societal: Analyzing Persona-Induced Bias in Multi-Agent Interactions (AAAI 2026 · similarity 0.88)
- Persona-Model Collapse in Emergent Misalignment (NeurIPS 2026 · similarity 0.88)
- ToMATO: Verbalizing the Mental States of Role-Playing LLMs for Benchmarking Theory of Mind (AAAI 2025 · similarity 0.88)
- StereoTales: A Multilingual Framework for Open-Ended Stereotype Discovery in LLMs (NeurIPS 2026 · similarity 0.88)
- Investigating and Mitigating Undesirable Biases in Large Language Models (AAAI 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.