Research map / Large language models / Language, safety and interpretability
Is Your Model Fairly Certain? Uncertainty-Aware Fairness Evaluation for LLMs
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes UCerF, an uncertainty-aware fairness metric based on perplexity-derived certainty and a linear scale of behavior preference, plus SynthBias, a 31,756-sample GPT-4o-generated and human-validated gender-occupation coreference dataset, and benchmarks ten open LLMs.
Problem
Accuracy-based fairness metrics such as Equalized Odds ignore model uncertainty, and existing gender-occupation coreference benchmarks like WinoBias are small, template-based, and too easy for modern LLMs.
Approach
Combine prediction correctness and perplexity-based certainty into a desirability score on a linear scale, and measure fairness as the expected disparity of desirability between minimal-pair groups (UCerF), alongside a new LLM-generated, human-verified benchmark (SynthBias).
Key results
- On SynthBias type2 intrinsic evaluation, Falcon-40B-Instruct ranks second on UCerF (0.793) and second on accuracy (0.840), giving the highest joint FP ranking.
- Mistral-7B-Instruct ranks 3rd on accuracy (0.799) and 4th on EO but drops to 8th under UCerF (0.678) due to confident biased predictions.
- Llama-3-70B-Instruct ranks third on WinoBias type1 but eighth on SynthBias type1, showing hidden bias exposed by the stricter ambiguity filter.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use SynthBias instead of WinoBias for type2 intrinsic fairness evaluation | LLM fairness evaluation with UCerF · Gender-occupation co-reference type2 | Accuracy | Hurt |
| Evaluate on SynthBias rather than WinoBias | LLM fairness evaluation with UCerF · Gender-occupation co-reference type2 | UCerF: 6% | Hurt |
| Use MCQ options rather than the intrinsic next-word task | LLM fairness evaluation · Type1 and type2 | UCerF | Helped |
| Account for uncertainty alongside correctness | Equalized Odds fairness evaluation · SynthBias type2; Falcon-40B-Instruct case study | Fairness assessment: 0.552 | Helped |
| Replace EO-only assessment with UCerF | Fairness evaluation of Mistral-7B-Instruct · WinoBias type2 | Fairness ranking: fifth -> eighth | Hurt |
| Evaluate fairness with UCerF rather than accuracy ranking | Pythia-1B · WinoBias type2 | Fairness ranking: tenth -> fifth | Helped |
| Use human-validated ambiguity in SynthBias type1 rather than WinoBias type1 | Llama-3-70B-Instruct · Type1 fairness | UCerF ranking: third -> eighth | Hurt |
| Use a more challenging SynthBias evaluation rather than WinoBias | Llama-3-70B-Instruct and Mixtral-8x7B-Instruct · Gender-occupation fairness | Fairness ranking: from first & second to third & fourth | Hurt |
| Use uncertainty-aware UCerF rather than EO for the same model behavior | Falcon-40B-Instruct · SynthBias type2, physician-nurse case study | Fairness disparity: 1 and 0 -> difference of 0.5; 0.797 | Helped |
Surprising findings
- Equalized Odds can overstate fairness relative to UCerF, e.g., a model with TPR gap 0.5 under EO was judged more fair under UCerF.
- Mixtral-8x7B-Instruct is fair under UCerF despite high confidence, so UCerF does not simply reward uncertainty.
Limitations
- Perplexity is used as the only uncertainty estimator; other estimators are not explored.
- Dataset is limited to binary gender pronouns and US-centric BLS occupations.
- Evaluation is limited to coreference resolution with a fixed prompt, and results may be sensitive to prompt wording.
Experimental setup
Datasets: WinoBias SynthBias
Benchmarks: WinoBias SynthBias
Metrics: accuracy equalized odds perplexity UCerF joint fairness-performance (FP)
Baselines: Equalized Odds WinoBias
Builds on: Equalized Odds WinoBias BBQ
Closest papers by meaning
- An Empirical Analysis of Uncertainty in Large Language Model Evaluations (ICLR 2025 · similarity 0.89)
- All Should Be Equal in the Eyes of LMs: Counterfactually Aware Fair Text Generation (AAAI 2024 · similarity 0.88)
- FairMT-Bench: Benchmarking Fairness for Multi-turn Dialogue in Conversational LLMs (ICLR 2025 · similarity 0.88)
- Benchmarking LLMs via Uncertainty Quantification (NeurIPS 2024 · similarity 0.88)
- Can LLMs Express Their Uncertainty? An Empirical Evaluation of Confidence Elicitation in LLMs (ICLR 2024 · similarity 0.87)
- AIM-Fair: Advancing Algorithmic Fairness via Selectively Fine-Tuning Biased Models with Contextual Synthetic Data (CVPR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.