Research map / Probabilistic and scientific modeling / Variational and generative sampling
Robust Evaluation Measures for Evaluating Social Biases in Masked Language Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes KLS and JSS, bias measures for masked language models that model stereotypical and anti-stereotypical pseudo-log-likelihood scores as Gaussians and compare them via KL and JS divergence, showing greater robustness on subsampled StereoSet and CrowS-Pairs.
Problem
Indicator-function-based bias measures for MLMs (comparing PLL scores of stereotypical vs anti-stereotypical sentences) are non-robust on limited datasets and ignore the magnitude and distribution of the scores.
Approach
Represent the stereotypical and anti-stereotypical PLL score sets as Gaussian distributions and build bias measures from KL divergence (KLS) and JS divergence with a standard-deviation penalty (JSS), instead of counting pairwise wins.
Key results
- On CrowS-Pairs under 30%-80% subsampling, KLS and JSS reproduce the full-dataset results at all sampling rates, while CPS and AUL flip conclusions at several rates.
- BERT receives the least-biased scores on both datasets across all measures, e.g., KLS 51.59 on StereoSet and 53.36 on CrowS-Pairs.
- KLS correlates with SSS at r=0.84, the lowest correlation among measures, attributed to the masking strategy.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace the indicator-based bias measure with KLS based on Gaussian PLL-score distributions | MLM bias evaluation on CrowS-Pairs · CrowS-Pairs subsets at 30%, 40%, 50%, 60%, 70%, and 80% sampling rates | consistency of bias evaluation across sampling rates | Helped |
| Replace the indicator-based bias measure with JSS based on Gaussian PLL-score distributions | MLM bias evaluation on CrowS-Pairs · CrowS-Pairs subsets at 30%, 40%, 50%, 60%, 70%, and 80% sampling rates | consistency of bias evaluation across sampling rates | Helped |
| Evaluate with CPS rather than the proposed distributional measures | MLM bias evaluation on CrowS-Pairs · CrowS-Pairs subsets at 30% and 60% sampling rates | consistency of bias evaluation across sampling rates | Hurt |
| Evaluate with AUL rather than the proposed distributional measures | MLM bias evaluation on CrowS-Pairs · CrowS-Pairs subsets at 30%, 40%, 50%, 60%, 70%, and 80% sampling rates | consistency of bias evaluation across sampling rates | Hurt |
Surprising findings
- The indicator-based measure gives a score of 50 for a model whose stereotypical and anti-stereotypical PLL gaps are strongly imbalanced, so it can call a clearly biased model ideal.
- Annotators disagreed on bias types for some sentence pairs, indicating ambiguity in the StereoSet and CrowS-Pairs labels.
Limitations
- Experiments are limited to three small-to-medium MLMs (BERT, RoBERTa, ALBERT) and two datasets.
- Gaussian assumption is checked only with a Shapiro-Wilk test and kernel density plots, and fails for BERT on StereoSet at the 0.01 level.
- Robustness evaluated via random subsampling of CrowS-Pairs only; the StereoSet development set is used because its test set is not public.
Experimental setup
Datasets: StereoSet CrowS-Pairs
Benchmarks: StereoSet CrowS-Pairs
Metrics: KLS JSS bias score (percentage of stereotype-preferred pairs) Pearson correlation mutual information
Baselines: StereoSet Score (SSS) CrowS-Pairs Score (CPS) All Unmasked Likelihood (AUL)
Builds on: StereoSet CrowS-Pairs All Unmasked Likelihood pseudo-log-likelihood
Closest papers by meaning
- The Devil is in the Neurons: Interpreting and Mitigating Social Biases in Language Models (ICLR 2024 · similarity 0.88)
- StereoTales: A Multilingual Framework for Open-Ended Stereotype Discovery in LLMs (NeurIPS 2026 · similarity 0.87)
- Biases Mitigation and Expressiveness Preservation in Language Models: A Comprehensive Pipeline (Student Abstract) (AAAI 2024 · similarity 0.87)
- SocialStigmaQA: A Benchmark to Uncover Stigma Amplification in Generative Language Models (AAAI 2024 · similarity 0.87)
- Investigating and Mitigating Undesirable Biases in Large Language Models (AAAI 2025 · similarity 0.86)
- Bias and Volatility: A Statistical Framework for Evaluating Large Language Model's Stereotypes and the Associated Generation Inconsistency (NeurIPS 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.