Research map / Large language models / Language, safety and interpretability
Bias and Volatility: A Statistical Framework for Evaluating Large Language Model's Stereotypes and the Associated Generation Inconsistency
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes the PVF statistical framework that decomposes LLM stereotype/discrimination risk into prejudice (bias) and volatility (context-dependent variance), estimated with context templates mined from natural text.
Problem
Existing LLM bias metrics treat stereotypes as systematic and ignore the randomness and context-sensitivity of generation, so models can show contradictory stereotypes across contexts.
Approach
Define discrimination risk over attribute-specific token likelihoods conditioned on context templates, then decompose it into a prejudice term (squared bias of the expected conditional probability) and a volatility term (variance across contexts), and report both per social group.
Key results
- For religion, Llama2 discrimination risk R = 0.1423 with prejudice 0.1380 and volatility 0.0043, while T5 has the highest R = 0.1895.
- For gender nurse profession, Llama2 has volatility 0.1386 versus prejudice 0.0848, indicating volatility dominates for pre-female professions.
- Model rankings for gender bias stay consistent across webpage, book, conversation and textbook context sources, though magnitudes vary.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Evaluate gender discrimination for atypical professions (nurse and stylist) | Llama2, GPT2, and BERT · Nurse and stylist | volatility risk (r^v): Llama2: 0.1386 (Nurse), 0.0504 (Stylist); GPT2: 0.0548 (Nurse), 0.0524 (Stylist); BERT: 0.0406 (Nurse) | Mixed |
| Use different context-mining data sources (book, conversation, textbook, webpage) | PVF gender-discrimination assessment · Gender discrimination across context sources | risk level | Mixed |
Surprising findings
- For pre-female professions such as nurse and stylist, LLMs show a pro-male bias contrary to societal stereotypes, with volatility risk well above the model average.
- Volatility risk is far smaller than prejudice risk in most measurements, though it can be large for some groups and models.
Limitations
- Excludes proprietary LLMs because their predictive distributions are not disclosed.
- Evaluation limited to a few open models and English-centric templates; context selection and data quality not thoroughly examined.
Experimental setup
Datasets: Toxic Tweets Dataset Offensive Language Dataset Toxicity Dataset Webpage, book, conversation, textbook text corpora (context template mining)
Metrics: discrimination risk R prejudice risk R^b / r^p volatility risk R^v / r^v
Baselines: BERT GPT-2 T5 Llama 2
Builds on: McCauley et al. stereotype conditional probability Gallegos et al. bias survey
Closest papers by meaning
- StereoTales: A Multilingual Framework for Open-Ended Stereotype Discovery in LLMs (NeurIPS 2026 · similarity 0.91)
- Examining Alignment of Large Language Models through Representative Heuristics: the case of political stereotypes (ICLR 2025 · similarity 0.88)
- Are Models Biased on Text without Gender-related Language? (ICLR 2024 · similarity 0.88)
- Investigating and Mitigating Undesirable Biases in Large Language Models (AAAI 2025 · similarity 0.88)
- SocialStigmaQA: A Benchmark to Uncover Stigma Amplification in Generative Language Models (AAAI 2024 · similarity 0.87)
- Prompting Fairness: Integrating Causality to Debias Large Language Models (ICLR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.