Research map / Large language models / Language, safety and interpretability
Investigating and Mitigating Undesirable Biases in Large Language Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
A doctoral consortium summary of research on undesirable biases in LLMs, covering social bias datasets, brand bias, persona effects on norm interpretation, and dual-process-based prompting to mitigate bias.
Problem
LLMs encode social, brand, cultural, and persona-linked biases that can perpetuate stereotypes and unfair outcomes, including in low-resource languages.
Approach
Summarizes a line of work that measures subtler social biases with template-based sentence completion, probes brand and nationality/persona biases, and mitigates social bias via prompting strategies inspired by dual process theory (System 1/System 2, CoT, debiasing, human personas).
Key results
- All four evaluated LLMs (GPT-4, Llama-2, PaLM-2, Mistral) show statistically significant biases on less-studied categories such as age, beauty, and institution (Kendall's tau test).
- Human persona, debiasing, System 2, and CoT prompting all tend to reduce social biases, with up to a 19 percent drop in stereotypical judgments.
- LLM-persona combinations across 193 nationality personas tend to favor Western European nations.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use a human persona, debiasing prompt, System 2 prompting, or chain-of-thought prompting | LLMs · Social bias detection datasets (two datasets; nine categories) | social bias | Helped |
| Combine a human persona, debiasing, System 2 prompting, and chain-of-thought prompting | LLMs · Social bias detection datasets | stereotypical judgments: up to a 19 percent drop | Helped |
| Use different combinations of human persona, debiasing, System 2, and chain-of-thought prompting | LLMs · Social bias detection datasets | social bias | Mixed |
| Evaluate age, beauty, beauty in profession, and institution-related associations | GPT-4, Llama-2, PaLM-2, and Mistral · Subtler social bias categories | bias (Kendall’s τ) | Hurt |
| Compare global and local brands | LLMs · Brand recommendations and associations | positive brand associations | Mixed |
| Recommend gifts for people in high-income countries | LLMs · Brand recommendations | gift luxury | Mixed |
| Use a country-of-origin context | LLMs · Brand recommendations | local brand preference | Helped |
| Assign a more socially desirable persona (for example, a thin person rather than a fat person) | LLMs · Social norm interpretation | interpretation accuracy | Helped |
| Assign different sociodemographic personas | LLMs · Social norm interpretation | interpretation consistency | Hurt |
| Assign nationality personas | LLMs · Country perceptions | regional favorability | Mixed |
Surprising findings
- Assigning a more socially desirable persona (e.g., thin person) yields more accurate social norm interpretation than a less desirable persona (e.g., fat person).
Limitations
- Summarizes work by the author; details of individual studies are not provided in this paper.
- Mitigation results are reported only as a summary of cited work.
Experimental setup
Datasets: template-generated sentence completion dataset (social bias) curated brand dataset (shoe, cloth, electronics, beverage) BanStereoSet two bias detection datasets spanning nine social bias categories nationality and persona norm datasets
Benchmarks: BanStereoSet two bias detection datasets (nine social bias categories)
Metrics: Kendall's tau correlation stereotypical judgment rate
Baselines: zero-shot CoT debiasing prompt
Builds on: chain-of-thought prompting StereoSet
Closest papers by meaning
- StereoTales: A Multilingual Framework for Open-Ended Stereotype Discovery in LLMs (NeurIPS 2026 · similarity 0.89)
- Biases Mitigation and Expressiveness Preservation in Language Models: A Comprehensive Pipeline (Student Abstract) (AAAI 2024 · similarity 0.88)
- UniBias: Unveiling and Mitigating LLM Bias through Internal Attention and FFN Manipulation (NeurIPS 2024 · similarity 0.88)
- Large Language Models are Geographically Biased (ICML 2024 · similarity 0.88)
- Bias and Volatility: A Statistical Framework for Evaluating Large Language Model's Stereotypes and the Associated Generation Inconsistency (NeurIPS 2024 · similarity 0.88)
- Debiasing Algorithm through Model Adaptation (ICLR 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.