Research map / Large language models / Language, safety and interpretability
The Silent Amplifier: In-Context Examples Fuel Bias in Large Language Models
Xinwei Guo, Jiashi Gao, Junlei Zhou, Jiaxin Zhang, Quanying Liu, Haiyan Wu, Xin Yao, Xuetao Wei
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use ICL example selection instead of zero-shot | LLMs · EEC-paraphrase | max gender/race bias: bias above zero-shot in many LLMs | Hurt |
| select examples for higher accuracy (similarity, DPP) | OPT-6.7B / GPT-J-6B · EEC-paraphrase | AvgGF / accuracy: acc 0.92 / 0.93 but max MaxFG still 0.20 / 0.17 (GPT-J-6B) | Mixed |
| add ReBE (bias-aware prompt embedding) to example selection | Random/Perplexity/Similarity/DPP · EEC-paraphrase | maximum gender bias: e.g. OPT-13B DPP AvgGF max 0.080 (-0.043), MaxFG max (-0.217) | Improved |
| add ReBE | example selection baselines · EEC-paraphrase | accuracy: DPP avg 0.874 (+0.009) | No clear effect |
| train with Lacc only instead of Ltotal (ablation) | ReBE (GPT-J-6B) · EEC-paraphrase | max AvgGF: 0.082 -> 0.089 (acc 0.84 -> 0.86) | Hurt |
| add Lacc to Lbias (ablation) | ReBE (GPT-J-6B) · EEC-paraphrase | accuracy: 0.26 -> 0.84 | Improved |
| add Lbias to Lacc (ablation) | ReBE (GPT-J-6B) · EEC-paraphrase | MaxTG max: 0.292 -> 0.221 | Improved |
| use ReBE instead of counterfactual or gender-balanced context | OPT-6.7B · EEC-paraphrase | AvgGF max: 0.132 / 0.125 -> 0.086 (Rand+ReBE) | Improved |
| combine ReBE with DPP selection | OPT-6.7B · EEC-paraphrase | accuracy and MaxTG max: MaxTG max 0.273 -> 0.250; acc 0.87 -> 0.87 | Improved |
| increase k-shot | ReBE (GPT-J-6B) · EEC-paraphrase | accuracy and bias | Mixed |
| vary number of virtual tokens | ReBE (GPT-J-6B) · EEC-paraphrase | bias | No clear effect |
Closest papers by meaning
- UniBias: Unveiling and Mitigating LLM Bias through Internal Attention and FFN Manipulation (NeurIPS 2024 · similarity 0.89)
- Is attention required for ICL? Exploring the Relationship Between Model Architecture and In-Context Learning Ability (ICLR 2024 · similarity 0.87)
- GistScore: Learning Better Representations for In-Context Example Selection with Gist Bottlenecks (ICML 2024 · similarity 0.87)
- In-context Vectors: Making In Context Learning More Effective and Controllable Through Latent Space Steering (ICML 2024 · similarity 0.87)
- Why Larger Language Models Do In-context Learning Differently? (ICML 2024 · similarity 0.87)
- Exact Conversion of In-Context Learning to Model Weights in Linearized-Attention Transformers (ICML 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
