Research map / Large language models / Language, safety and interpretability
BiasBusters: Uncovering and Mitigating Tool Selection Bias in Large Language Models
Thierry Blankenstein, Jialin Yu, Zixuan Li, Vassilis Plachouras, Sunando Sengupta, Philip Torr, Yarin Gal, Alasdair Paren, Adel Bibi
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| apply subset-filter then uniform-sampling mitigation | GPT-4.1 mini tool selection · bias benchmark | delta_model (lower better): 0.380 -> 0.094 | Improved |
| apply mitigation | GPT-4.1 mini · bias benchmark | delta_pos: 0.422 -> 0.079 | Improved |
| biased continued pre-training on one endpoint | Qwen3-8B · Language Identification cluster | selection share of exposed endpoint: 0.006 -> 0.128 | Improved |
| raise temperature from 0 to 2 | GPT-3.5 · bias benchmark | delta_model: 0.350 -> 0.285 | Improved |
| raise top-p from 0.7 to 1.0 | GPT-3.5 · bias benchmark | delta_model: 0.346 -> 0.340 | No clear effect |
| increase model size (Qwen3 1.7B to 235B) | Qwen3 · bias benchmark | delta_model | Improved |
| cyclic vs random API ordering | GPT-3.5 · bias benchmark | selection distribution | No clear effect |
| reword or restructure system prompt | GPT-3.5 · bias benchmark | selection distribution | Mixed |
| scramble descriptions and parameters of tools | Gemini 2.5 Flash · metadata perturbation | TV distance from base: 0.450 | Improved |
| swap descriptions of most- and least-selected tools | Gemini 2.5 Flash · metadata perturbation | TV distance from base: 0.338 | Improved |
| scramble tool names only | Gemini / GPT-3.5 · metadata perturbation | selection shift | No clear effect |
| add random forest instead of linear regression for predicting selection | feature analysis · 50 APIs per model | predictive performance | No clear effect |
Closest papers by meaning
- MetaTool Benchmark for Large Language Models: Deciding Whether to Use Tools and Which to Use (ICLR 2024 · similarity 0.90)
- Auto-Search and Refinement: An Automated Framework for Gender Bias Mitigation in Large Language Models (NeurIPS 2025 · similarity 0.88)
- ToolSearcher: Optimizing Tool Selection at Scale via Reinforcement Learning (NeurIPS 2026 · similarity 0.88)
- Biases Mitigation and Expressiveness Preservation in Language Models: A Comprehensive Pipeline (Student Abstract) (AAAI 2024 · similarity 0.87)
- Investigating and Mitigating Undesirable Biases in Large Language Models (AAAI 2025 · similarity 0.87)
- Prompting Fairness: Integrating Causality to Debias Large Language Models (ICLR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
