Research map / Large language models / Language, safety and interpretability
Fairness-Aware Structured Pruning in Transformers
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes FASP, a training-free structured attention-head pruning method that removes heads harmful to fairness (toxicity-based social bias) while keeping heads critical for perplexity, reducing gender bias across six pretrained LMs without fine-tuning.
Problem
Existing attention-head pruning methods for LLMs optimize only for language modeling performance and ignore fairness/social bias, and often assume costly fine-tuning after pruning.
Approach
Score each attention head by its effect (via knockout) on both WikiText-2 perplexity and toxicity-based bias across demographic subgroups; keep the top gamma fraction of heads important for perplexity, then prune heads with the largest bias contribution among the rest, with no fine-tuning.
Key results
- FASP reduces gender bias by 19%, 19.5%, 39.5%, 34.7%, 23%, and 8% for DistilGPT-2, GPT-2, GPT-Neo (two sizes), GPT-J, and Llama 2 respectively, relative to the unpruned model, with only slight perplexity increase.
- FASP is the only evaluated pruning method that consistently reduces gender bias without perplexity overshooting; gamma of 0.3 was selected for five of six models.
- Pruning heads for gender bias also reduces sexual orientation, race, and nationality biases in most cases, while religion bias is mixed (increases for DistilGPT-2).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| apply FASP pruning | DistilGPT-2 · Gender bias | gender bias reduction: 19\% | Helped |
| apply FASP pruning | GPT-2 · Gender bias | gender bias reduction: 19.5\% | Helped |
| apply FASP pruning | GPT-Neo 125M · Gender bias | gender bias reduction: 39.5\% | Helped |
| apply FASP pruning | GPT-Neo 1.3B · Gender bias | gender bias reduction: 34.7\% | Helped |
| apply FASP pruning | GPT-J · Gender bias | gender bias reduction: 23\% | Helped |
| apply FASP pruning | Llama 2 · Gender bias | gender bias reduction: 8\% | Helped |
| prune heads using performance scores only | language models evaluated · Gender bias | gender bias | No clear effect |
| prune heads by magnitude | GPT-Neo and Llama 2 · WikiText-2 | perplexity | Hurt |
| prune heads randomly | language models evaluated · WikiText-2 | perplexity | Hurt |
| prune gender-bias-impactful heads | FASP · Sexual orientation, race, and nationality bias | bias | Helped |
| prune gender-bias-impactful heads | FASP on GPT-2 and GPT-Neo · Religion bias | bias | Helped |
| prune gender-bias-impactful heads | FASP on DistilGPT-2 · Religion bias | bias | Hurt |
Surprising findings
- Existing head pruning methods (magnitude, gradient) do not improve fairness and can increase bias.
- Pruning heads for gender bias can also reduce other social biases because head bias scores are positively correlated across groups.
Limitations
- The same pruning technique could be used to amplify bias by targeting bias-counteracting heads.
- Relies on a BERT-based toxicity classifier that may itself be biased or inaccurate.
- Assumes additive head effects despite non-linearity of the model.
Experimental setup
Datasets: HolisticBias WikiText-2
Benchmarks: HolisticBias WikiText-2
Metrics: toxicity-based bias score (gender) perplexity on WikiText-2
Baselines: magnitude pruning gradient pruning random pruning fairness only pruning performance only pruning
Builds on: magnitude pruning gradient-based head pruning Are Sixteen Heads Really Better Than One? BOLD Holistic bias
Closest papers by meaning
- Beyond Linear Approximations: A Novel Pruning Approach for Attention Matrix (ICLR 2025 · similarity 0.88)
- Debiasing Algorithm through Model Adaptation (ICLR 2024 · similarity 0.87)
- Evaluating the Efficacy of Prompting Techniques for Debiasing Language Model Outputs (Student Abstract) (AAAI 2024 · similarity 0.87)
- Biases Mitigation and Expressiveness Preservation in Language Models: A Comprehensive Pipeline (Student Abstract) (AAAI 2024 · similarity 0.87)
- LIDAO: Towards Limited Interventions for Debiasing (Large) Language Models (ICML 2024 · similarity 0.87)
- FairMT-Bench: Benchmarking Fairness for Multi-turn Dialogue in Conversational LLMs (ICLR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.