Research map / Large language models / Language, safety and interpretability
UniBias: Unveiling and Mitigating LLM Bias through Internal Attention and FFN Manipulation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Identifies biased FFN vectors and attention heads inside Llama-2 via vocabulary projection and three criteria, then masks them at inference to improve ICL accuracy and robustness to prompt brittleness.
Problem
LLM label bias in in-context learning causes prompt brittleness (sensitivity to example choice, order, and formatting), but prior debiasing methods only adjust outputs externally and the internal mechanisms are unexplored.
Approach
Project each FFN value vector and attention head output into vocabulary space (logit lens) to get label logits, flag components that are label-relevant, label-biased, and low-variance across samples, then mask them at inference; thresholds are chosen by grid search on a small labeled support set.
Key results
- With Llama-2 7b one-shot, UniBias averages 70.46 vs 67.07 for standard ICL and 68.70 for the best baseline DC (+1.76 over DC).
- With Llama-2 13b, UniBias averages 75.51 vs 72.54 ICL and 73.06 for CC.
- Under prompt-design perturbations, standard ICL ranges 8% to 26% while UniBias varies by less than 4%.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Apply UniBias instead of standard ICL | Llama-2 7b · 12-dataset average, 1-shot ICL | accuracy: 67.07 -> 70.46 | Helped |
| Apply UniBias instead of DC | Llama-2 7b · 12-dataset average, 1-shot ICL | accuracy: 68.70 -> 70.46 | Helped |
| Apply UniBias instead of standard ICL | Llama-2 13b · 12-dataset average, 1-shot ICL | accuracy: 72.54 -> 75.51 | Helped |
| Apply UniBias instead of CC | Llama-2 13b · 12-dataset average, 1-shot ICL | accuracy: 73.06 -> 75.51 | Helped |
| Remove biased attention heads (FFN-only) (ablation) | UniBias · Across 12 datasets | accuracy | Mixed |
| Remove biased FFN vectors (attention-only) (ablation) | UniBias · Across 12 datasets | accuracy | Mixed |
| Mask commonly identified biased attention heads across tasks instead of identifying components per task (ablation) | Llama-2 7b · SST2, MMLU, COPA, RTE, MR, TREC average | accuracy: 70.84 -> 73.98 | Helped |
Surprising findings
- Masking a small set of frequently biased attention heads shared across tasks (common biased components) already improves average accuracy from 70.84 to 73.98 on six datasets without per-task identification.
Limitations
- Relies on grid search with a small labeled support set (20 per class), which limits applicability when labels are scarce.
- Biased components are identified per ICL prompt, making the method prompt-specific rather than global.
Experimental setup
Datasets: SST-2 SST-5 MR CR AGNews TREC MNLI RTE ARC-Challenge MMLU COPA WiC
Benchmarks: SST-2 SST-5 MR CR AGNews TREC MNLI RTE ARC-Challenge MMLU COPA WiC
Metrics: accuracy standard deviation over five seeds average accuracy
Baselines: standard ICL Contextual Calibration Domain-Context Calibration Prototypical Calibration
Builds on: logit lens residual stream view of transformers contextual calibration
Closest papers by meaning
- Supervised Knowledge Makes Large Language Models Better In-context Learners (ICLR 2024 · similarity 0.89)
- Unveiling and Harnessing Hidden Attention Sinks: Enhancing Large Language Models without Training through Attention Calibration (ICML 2024 · similarity 0.89)
- Wisdom is Knowing What not to Say: Hallucination-Free LLMs Unlearning via Attention Shifting (NeurIPS 2025 · similarity 0.88)
- Investigating and Mitigating Undesirable Biases in Large Language Models (AAAI 2025 · similarity 0.88)
- Language Bias in LVLMs: From In-Depth Analysis to Simple and Effective Mitigation (ICML 2026 · similarity 0.88)
- Remarkable Robustness of LLMs: Stages of Inference? (NeurIPS 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.