Research map / Large language models / Language, safety and interpretability
SocialStigmaQA: A Benchmark to Uncover Stigma Amplification in Generative Language Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces SocialStigmaQA, a ~10K-prompt yes/no/can't-tell QA benchmark testing generative LMs (Flan-T5-XXL, Flan-UL2) for social bias against 93 US stigmas across prompt styles and decoding strategies.
Problem
Existing social bias benchmarks focus on protected demographic attributes such as race and gender, leaving stigmatized conditions (e.g., criminal record, obesity, HIV) largely unaudited in generative language models.
Approach
A QA benchmark built from 37 hand-curated social-situation templates instantiated with 93 documented stigmas, with four prompt styles (original, positive bias, doubt bias, no-stigma) so that bias and robustness can be measured, and with yes/no answer choices so that either answer can be the biased one.
Key results
- Proportion of socially biased output ranges from 45% to 59% across decoding strategies and prompt styles for Flan-T5 (greedy 0.450 and CoT 0.492 for Flan-T5; 0.592 greedy and 0.483 CoT for Flan-UL2).
- Positive-bias prompt style reduced biased answers across all experiments (e.g., Flan-T5 greedy 'no'-biased original 84% vs positive bias 47%).
- Chain-of-thought had mixed effects: it reduced bias for 'no'-biased prompts (0.692 to 0.535) but raised bias for 'yes'-biased prompts (0.052 to 0.423) for Flan-T5.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add chain-of-thought via nucleus sampling | Flan-T5 · SocialStigmaQA prompts where the biased answer is no | biased-answer proportion: 0.692 to 0.535 | Helped |
| Add chain-of-thought via nucleus sampling | Flan-T5 · SocialStigmaQA prompts where the biased answer is yes | biased-answer proportion: 0.052 to 0.423 | Hurt |
| Add chain-of-thought via nucleus sampling | Flan-UL2 · SocialStigmaQA prompts where the biased answer is no | biased-answer proportion: 0.855 to 0.583 | Helped |
| Add chain-of-thought via nucleus sampling | Flan-UL2 · SocialStigmaQA prompts where the biased answer is yes | biased-answer proportion: 0.160 to 0.319 | Hurt |
| Add positive-bias text to the prompt (ablation) | Flan-T5 and Flan-UL2 · SocialStigmaQA | biased-answer proportion | Helped |
| Add positive-bias text to the prompt (ablation) | Flan-T5, greedy; biased answer no · SocialStigmaQA | biased-answer proportion: 0.844 to 0.470 | Helped |
| Add positive-bias text to the prompt (ablation) | Flan-T5, nucleus CoT; biased answer yes · SocialStigmaQA | biased-answer proportion: 0.492 to 0.303 | Helped |
| Add positive-bias text to the prompt (ablation) | Flan-UL2, greedy; biased answer no · SocialStigmaQA | biased-answer proportion: 93% to 67% | Helped |
| Add positive-bias text to the prompt (ablation) | Flan-UL2, nucleus CoT; biased answer no · SocialStigmaQA | biased-answer proportion: 64% to 42% | Helped |
| Add doubt-bias text to the prompt (ablation) | Flan-T5 and Flan-UL2 · SocialStigmaQA | biased-answer proportion | No clear effect |
| Add doubt-bias text to the prompt (ablation) | Flan-T5, nucleus CoT; biased answer yes · SocialStigmaQA | biased-answer proportion: 49% to 47% | No clear effect |
| Remove mention of the stigma (no-stigma control) (ablation) | Flan-T5 · SocialStigmaQA prompts where yes indicates bias | yes-answer proportion: 0 times to 36% | Mixed |
Surprising findings
- Flan-T5 with nucleus sampling CoT answered 'yes' 36% of the time on no-stigma prompts where 'yes' is biased, while greedy decoding produced 'yes' 0 times, showing the model's default answer propensity shifts with decoding.
- Doubt-bias text (e.g., 'I don't know what to do') did not substantially change biased-answer rates, indicating robustness, while CoT outputs often contradicted their final answers.
Limitations
- English-only and US-centric stigmas; cultural transfer not evaluated.
- Evaluation of open-ended CoT text is unsolved; manual annotation covered only 600 responses and CoT often misaligned with final answer.
- No-stigma control shows model yes/no propensity, and it is unclear how to adjust bias scores for this.
Experimental setup
Datasets: SocialStigmaQA
Benchmarks: SocialStigmaQA
Metrics: proportion of biased answers answer distribution (yes/no/can't tell)
Baselines: Flan-UL2
Builds on: chain-of-thought prompting BBQ UnQover
Closest papers by meaning
- The Devil is in the Neurons: Interpreting and Mitigating Social Biases in Language Models (ICLR 2024 · similarity 0.90)
- Mitigating Social Bias in Large Language Models: A Multi-Objective Approach Within a Multi-Agent Framework (AAAI 2025 · similarity 0.89)
- StereoTales: A Multilingual Framework for Open-Ended Stereotype Discovery in LLMs (NeurIPS 2026 · similarity 0.89)
- Prompting Fairness: Integrating Causality to Debias Large Language Models (ICLR 2025 · similarity 0.89)
- Probing Social Bias in Labor Market Text Generation by ChatGPT: A Masked Language Model Approach (NeurIPS 2024 · similarity 0.88)
- Are Models Biased on Text without Gender-related Language? (ICLR 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.