Research map / Large language models / Language, safety and interpretability
Bias measurement and mitigation
Measure and reduce gender, geographic, and implicit bias in LLMs.
59 accepted papers at ICML, NeurIPS, ICLR, CVPR and AAAI (2016–2026) · an approach in Language, safety and interpretability, keyword cluster “safety · jailbreak · harmful”
State History Tried Checklist Preprints Who Export
Where it stands
Not enough benchmark data Fewer than four papers report the same benchmark, so there is no record to follow.
15 papers in 2023–24 and 38 in 2025–26 (at ICML, NeurIPS, ICLR, CVPR and AAAI); its share of the topic held steady, from 2.1% to 2.1%.
How this is decided
A benchmark is tracked when at least four papers of this approach report their own method on the same dataset and metric in at least two different years. Its record is the best number reported up to 2024 compared with the best reported in 2025–26: still improving means at least 1% better, flat means at most 0.2% better with at least two papers from 2025–26 trying. Numbers on a different scale from the rest (a fraction among percentages) and far outliers are left out. Results come from different papers with different setups, so they are signals, not head-to-head comparisons. The share of papers is counted at ICML, NeurIPS, ICLR, CVPR and AAAI.
How it got here
Most cited paper of each year
- 2016 · Discovering User Attribute Stylistic Differences via Paraphrasing (AAAI 2016 · 35 citations)
- 2019 · Assessing Social and Intersectional Biases in Contextualized Word Representations (NeurIPS 2019 · 67 citations)
- 2020 · FuzzE: Fuzzy Fairness Evaluation of Offensive Language Classifiers on African-American English (AAAI 2020 · 22 citations)
- 2021 · Towards Understanding and Mitigating Social Biases in Language Models (ICML 2021 · 124 citations)
- 2023 · Whose Opinions Do Language Models Reflect? (ICML 2023 · 95 citations)
- 2024 · Large Language Models are Geographically Biased (ICML 2024 · 13 citations)
- 2025 · Bias Unveiled: Investigating Social Bias in LLM-Generated Code (AAAI 2025 · 17 citations)
- 2026 · Adaptive Generation of Bias-Eliciting Questions for LLMs (ICML 2026 · 0 citations)
Earliest papers here
- Discovering User Attribute Stylistic Differences via Paraphrasing (AAAI 2016 · 35 citations)
- Beyond Exchangeability: The Chinese Voting Process (NeurIPS 2016 · 1 citation)
- Assessing Social and Intersectional Biases in Contextualized Word Representations (NeurIPS 2019 · 67 citations)
What has been tried
35 papers here report 325 measured changes (a component added, removed or swapped, compared with a baseline): helped 48%, no clear effect 12%, hurt 27%, mixed 13%. Papers mostly publish changes that helped, so "helped" is over-represented.
Ablations in recent papers
| What was changed | Measured on | Result | Effect | Paper |
|---|---|---|---|---|
| use Solar-feedback iterative prompting for three rounds | Solar social-bias dataset · CBS overall (%) | 60.58 -> 8.77 | Helped | Bias Unveiled: Investigating Social Bias in LLM-Generated Code (AAAI 2025 · 17 citations) |
| remove coordinates from the prompt | Population Density · Spearman's ρ | 0.73 -> 0.72 | No clear effect | Large Language Models are Geographically Biased (ICML 2024 · 13 citations) |
| Append task-agnostic debiasing instructions such as don't refuse, no stereotypes, or treat human | reasoning datasets · relative accuracy drop between personas | No clear effect | Bias Runs Deep: Implicit Reasoning Biases in Persona-Assigned LLMs (ICLR 2024 · 12 citations) | |
| Add positive-bias text to the prompt | SocialStigmaQA · biased-answer proportion | Helped | SocialStigmaQA: A Benchmark to Uncover Stigma Amplification in Generative Language Models (AAAI 2024 · 6 citations) | |
| Add the counteract-selection-bias prompt to Fact Only | WinoBias Type I Anti · TF error rate | 18.86 -> 10.58 | Helped | Prompting Fairness: Integrating Causality to Debias Large Language Models (ICLR 2025 · 5 citations) |
| Replace bert-base-uncased with DistilBERT | Human validation and LLM correlation plots · bias evaluation consistency | No clear effect | Probing Social Bias in Labor Market Text Generation by ChatGPT: A Masked Language Model Approach (NeurIPS 2024 · 4 citations) | |
| Increase certification sample count from fewer than 50 to 50 | BOLD prompt sets, Mistral-Instruct-v0.2 7B · tightness of certification bounds | Helped | Certifying Counterfactual Bias in LLMs (ICLR 2025 · 3 citations) | |
| Deduplicate pretraining data | USE 5, USE 10, USE 20, Winobias, and Winogender · Unstereo Score | 6.73; -17.14; 9.35 | Mixed | Are Models Biased on Text without Gender-related Language? (ICLR 2024 · 2 citations) |
| Use RAZOR's surface shortcut space instead of model token embeddings | FEVER · accuracy | Helped | RAZOR: Sharpening Knowledge by Cutting Bias with Unsupervised Text Rewriting (AAAI 2025 · 1 citation) | |
| use variational inference instead of MLE for VIRT | Concurrent validity · Overall | 0.4876 -> 0.8753 | Helped | Raising the Bar: Investigating the Values of Large Language Models via Generative Evolving Testing (ICML 2025 · 0 citations) |
| Add auto-searched Fairwords rather than using ChatGPT-generated instructions directly | GA-test · win rate | 50.64% -> 54.23% | Helped | Auto-Search and Refinement: An Automated Framework for Gender Bias Mitigation in Large Language Models (NeurIPS 2025 · 0 citations) |
| Increase the hybrid-prior mixing parameter to alpha=1.0 | Specified female software engineers · stability of statistically significant words | Hurt | More of the Same: Persistent Representational Harms Under Increased Representation (NeurIPS 2025 · 0 citations) |
What reviewers will expect
Datasets and baselines named in the 21 papers of this approach from 2025–2026 that have a full-text summary. Leaving out the common ones is a frequent review question.
Datasets
- BBQ · 5 of 21 papers
- RedditBias · 3 of 21 papers
- CrowS-Pairs · 2 of 21 papers
- MMLU · 2 of 21 papers
- WinoBias · 2 of 21 papers
- BOLD · 2 of 21 papers
Baselines
- zero-shot CoT · 2 of 21 papers
- Mistral-7B · 2 of 21 papers
- Marked Personas · 2 of 21 papers
Recent preprints
Loading recent arXiv preprints…
Who publishes on it
Institutions on the most papers of this approach since 2023 (of 53 papers; from OpenAlex and the papers' affiliations).
- Stanford University · 5 papers
- Microsoft · 3 papers
- IBM · 3 papers
- Microsoft Research Asia · 3 papers
- Carnegie Mellon University · 2 papers
- Peking University · 2 papers
- Chinese Academy of Sciences · 2 papers
- The Chinese University of Hong Kong · 2 papers
- University of Alberta · 2 papers
- City University of Hong Kong · 2 papers
Use it
Download BibTeX of 29 key papersOpen the topic in the interactive map
Get alerts. Describe your idea in scime atlas and save it (free scime account): you get a weekly email when new preprints land close to it.
Related approaches
Same keyword cluster
- Uncertainty quantification for LLMs (136 papers)
- Statistical frameworks for LLM evaluation (124 papers)
- Jailbreak elicitation and risk (21 papers)
Elsewhere in the topic
- LLM unlearning and benchmarks (298 papers)
- LLM pruning and scaling data (285 papers)
- Controlled text generation (220 papers)
- Faithfulness of LLM explanations (169 papers)
- Data and evaluation methodology (136 papers)
- Pruning and circuits in LLMs (135 papers)
- Safety alignment of LLMs (123 papers)
- Linguistic encoding in LLM representations (123 papers)
Benchmark numbers, ablations and baseline numbers are extracted by AI from each paper's full text and can be wrong; the paper is the authority. Data: results index of 2026-11-02, citation counts from OpenAlex (2026-10-09).