Research map / Large language models / Language, safety and interpretability
Prompting Fairness: Integrating Causality to Debias Large Language Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a causality-guided prompting framework (Dual Directional Prompting and related strategies) that nudges LLMs toward fact-based reasoning and away from social-salient cues to debias decisions under black-box access.
Problem
LLM decisions depend on social category information (gender, race, age, etc.) through biased shortcuts; closed-source LLMs can only be debiased via prompting, so a principled prompting approach is needed.
Approach
Causal modeling of how social information reaches LLM decisions through selection mechanisms; three prompting strategies (nudge toward social-agnostic fact, counteract historical selection bias with alternative facts, nudge away from social-salient text) combined into Dual Directional Prompting (DDP).
Key results
- On WinoBias Type II with GPT-4, DDP reduces the pro/anti gap to 0.13% (from 1.91% default).
- On WinoBias Type I with GPT-4, DDP gives 94.57% anti accuracy and a 2.17% gap versus 15.47% gap for default.
- On BBQ disambiguated setting with GPT-4, DDP achieves highest accuracy on 8 of 9 social categories, e.g., 97.58% on SES vs 68.36% default.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use DDP instead of ICL (16-shot) | GPT-4 · WinoBias Type I | gender gap: 9.23 -> 2.17 | Helped |
| Use DDP instead of ICL (16-shot) | GPT-4 · WinoBias Type II | gender gap: 1.52 -> 0.13 | Helped |
| Use DDP instead of ICL (8-shot) | GPT-4 · BBQ | accuracy | Mixed |
| Add the counteract-selection-bias prompt to Fact Only (ablation) | GPT-3.5 · WinoBias Type I Anti | TF error rate: 18.86 -> 10.58 | Helped |
| Add the fact-based-reasoning prompt to Counteract Only (ablation) | GPT-3.5 · WinoBias Type I Anti | TF error rate: 32.02 -> 10.58 | Helped |
| Add the counteract-selection-bias prompt to Fact Only (ablation) | GPT-4 · WinoBias Type I Anti | TF error rate: 5.29 -> 1.76 | Helped |
| Add the fact-based-reasoning prompt to Counteract Only (ablation) | GPT-4 · WinoBias Type I Anti | TF error rate: 25.78 -> 1.76 | Helped |
Surprising findings
- Counteract-only prompting (stating groups are equally likely) substantially reduces performance, while fact-only prompting performs quite well.
- Zero-shot CoT marginally reduces bias but lowers accuracy for GPT-3 and GPT-3.5.
Limitations
- Some WinoBias examples are ambiguous; 60 removed after human annotation, and the Type II shortcut may inflate small gaps.
- Effectiveness of DDP depends on the LLM's world knowledge and capability, so gains are tied to model strength.
- Theoretical guarantee relies on an assumption that a well-aligned LLM internalizes training dependence patterns, which is not verified.
Experimental setup
Datasets: WinoBias BBQ Discrim-Eval
Benchmarks: WinoBias BBQ Discrim-Eval
Metrics: accuracy pro-anti accuracy gap
Baselines: Default prompting ICL with contrastive examples Zero-shot CoT
Builds on: chain-of-thought prompting in-context learning with counterfactual examples
Closest papers by meaning
- Distributive Fairness in Large Language Models: Evaluating Alignment with Human Values (NeurIPS 2025 · similarity 0.90)
- Causal Prompting: Debiasing Large Language Model Prompting Based on Front-Door Adjustment (AAAI 2025 · similarity 0.90)
- Evaluating the Efficacy of Prompting Techniques for Debiasing Language Model Outputs (Student Abstract) (AAAI 2024 · similarity 0.90)
- Biases Mitigation and Expressiveness Preservation in Language Models: A Comprehensive Pipeline (Student Abstract) (AAAI 2024 · similarity 0.89)
- SocialStigmaQA: A Benchmark to Uncover Stigma Amplification in Generative Language Models (AAAI 2024 · similarity 0.89)
- Debiasing Algorithm through Model Adaptation (ICLR 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.