Research map / Large language models / AI and society
Large Language Models Develop Novel Social Biases Through Adaptive Exploration
Addison J. Wu, Ryan Liu, Xuechunzi Bai, Thomas L. Griffiths
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add chain-of-thought prompting (ablation) | frontier LLMs on iterative hiring task · Hiring task (p=0.9) | SI: Claude 4 Sonnet 1.787 -> 1.660 | No clear effect |
| increase temperature to 1.5 (ablation) | frontier LLMs (direct prompting) · Hiring task (p=0.9) | SI: Qwen 2.5-72B SI of 0.91 (p = 0.04) | No clear effect |
| compress context into running count summary (ablation) | GPT-4o, Claude 4 Sonnet, Llama 4 Maverick with CoT · Hiring task (p=0.9) | SI: SI of 1.39, 1.25, and 1.09 | No clear effect |
| replace job cover story with abstract options and contexts (ablation) | GPT-4o and Llama 4 Maverick · Abstract bandit setting | entropy of demographic-wise hiring distribution: GPT-4o: 0.62 (abstract), 1.90 (original); Llama 4 Maverick: 0.42 (abstract), 1.64 (original) | Hurt |
| lower success probability from 0.9 to 0.1 (ablation) | frontier LLMs · Hiring task | SI: Llama 4 Maverick CoT 1.257 -> 0.316; Claude 4 Sonnet CoT 1.660 -> 0.166 | Improved |
| use elicited realistic job-wise success probabilities instead of 0.1 (ablation) | GPT-4o · Hiring task | SI: SIs of 0.82 for direct and 0.60 for CoT | Mixed |
| remove gamified points; use refugee resettlement and conscription settings (ablation) | five frontier LLMs · Refugee resettlement / conscription | SI: average SIs of 1.13 and 1.26 | No clear effect |
| add age and education features (ablation) | frontier LLMs in refugee resettlement · Refugee resettlement | SI: mean SI reductions of 0.43, 0.59, and 0.70 for age, education, and both | Improved |
| add less salient features (hair color, tattoo shape) (ablation) | frontier LLMs in refugee resettlement · Refugee resettlement | SI: mean reductions in SI of 0.25, 0.44, and 0.42 for hair color, tattoo shape, and both | Mixed |
| extend hiring from 40 to 80 rounds with explicit notice (ablation) | GPT-4o and Gemini 2.5 Flash with unequal group success rates · Best-group identification accuracy | accuracy of identifying best group (%): 26.3% -> 26.2% | No clear effect |
| add explicit diversity bonus to the objective (ablation) | frontier LLMs on hiring task · Hiring task | SI | Improved |
| add fairness, internal-value, or shared-norm prompt steers (ablation) | frontier LLMs on hiring task · Hiring task | SI | Mixed |
Closest papers by meaning
- Large Language Models Think Too Fast To Explore Effectively (NeurIPS 2025 · similarity 0.89)
- Bias Amplification in Language Model Evolution: An Iterated Learning Perspective (NeurIPS 2024 · similarity 0.89)
- Prompting Fairness: Integrating Causality to Debias Large Language Models (ICLR 2025 · similarity 0.88)
- Distributive Fairness in Large Language Models: Evaluating Alignment with Human Values (NeurIPS 2025 · similarity 0.88)
- Can large language models explore in-context? (NeurIPS 2024 · similarity 0.88)
- LLM Agents Can Be Choice-Supportive Biased Evaluators: An Empirical Study (AAAI 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
