Research map / Large language models / Language, safety and interpretability
Bias Out-of-the-Box: An Empirical Analysis of Intersectional Occupational Biases in Popular Generative Language Models
Hannah Rose Kirk, Yennie Jun, Filippo Volpin, Haider Iqbal, Elias Benussi, Frederic A Dreyer, Aleksandar Shtedritski, Yuki Asano
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Prompt GPT-2 with woman rather than man | GPT-2 Small · Occupational prompts | jobs accounting for 50% of outputs: 16 -> 8 | Hurt |
| Prompt GPT-2 with woman rather than man | GPT-2 Small · Occupational prompts | jobs accounting for 90% of outputs: 66 vs 43 | Hurt |
| Add the woman dummy to the regression | GPT-2 occupational predictions · Occupational prompts across intersections | McFadden R²: +3.3% | Improved |
| Add gender-category interaction terms | GPT-2 occupational predictions · Occupational prompts across intersections | McFadden R²: +0.4% | Improved |
| Compare GPT-2 predictions with US labor-market distributions | GPT-2 · US occupational data, baseline woman | Kendall-Tau: 0.628 | Improved |
| Use GPT-2 predicted shares instead of US labor shares | Occupations with less than 25% women-share · US occupational data | difference in women-share: +8.7% | Mixed |
| Use GPT-2 predicted shares instead of US labor shares | Occupations with more than 75% women-share · US occupational data | difference in women-share: -6.5% | Mixed |
| Prompt with explicit demographic intersections rather than continental-origin names | GPT-2 · Occupational associations | intersectional effect on returned jobs | Mixed |
| Set top_k to 1 rather than the default 50 (ablation) | GPT-2 Small · Baseline man and woman occupational prompts | unique jobs: 228 -> 1; 250 -> 1 | Hurt |
| Set top_k to 10 rather than the default 50 (ablation) | GPT-2 Small · Baseline man and woman occupational prompts | unique jobs: 228 -> 51; 250 -> 30 | Hurt |
| Set top_k to 100 rather than the default 50 (ablation) | GPT-2 Small · Baseline man and woman occupational prompts | unique jobs: 228 -> 123; 250 -> 126 | Mixed |
| Set top_k to 500 rather than the default 50 (ablation) | GPT-2 Small · Baseline man and woman occupational prompts | unique jobs: 228 -> 233; 250 -> 204 | Mixed |
Closest papers by meaning
- Are Models Biased on Text without Gender-related Language? (ICLR 2024 · similarity 0.91)
- Probing Social Bias in Labor Market Text Generation by ChatGPT: A Masked Language Model Approach (NeurIPS 2024 · similarity 0.91)
- Bias and Volatility: A Statistical Framework for Evaluating Large Language Model's Stereotypes and the Associated Generation Inconsistency (NeurIPS 2024 · similarity 0.90)
- More of the Same: Persistent Representational Harms Under Increased Representation (NeurIPS 2025 · similarity 0.90)
- SocialStigmaQA: A Benchmark to Uncover Stigma Amplification in Generative Language Models (AAAI 2024 · similarity 0.89)
- SocialCounterfactuals: Probing and Mitigating Intersectional Social Biases in Vision-Language Models with Counterfactual Examples (CVPR 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
