Research map / Large language models / Language, safety and interpretability

Bias Unveiled: Investigating Social Bias in LLM-Generated Code

AAAI 2025 · Large language models › Language, safety and interpretability · approach: Bias measurement and mitigation · 17 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes Solar, a metamorphic-testing fairness framework with a 343-task dataset to measure social bias in LLM-generated code, and tests prompting-based mitigation (CoT, role-play CoT, iterative feedback prompting) on four code LLMs.

Problem

Social biases in code generated by LLMs for human-centered tasks (e.g., eligibility or suitability decisions based on demographics) are not evaluated or mitigated, and existing benchmarks lack realistic scenarios and severity quantification.

Approach

Generate code prompts and executable metamorphic test cases from a DSL (textX) task definition, mutating one protected attribute at a time, and use the failing-test results both to quantify bias (CBS, BLS) and as iterative feedback to the LLM to reduce bias.

Key results

• All four subject LLMs show severe bias; GPT-3.5-turbo-0125 has the highest overall CBS at 60.58% while CodeLlama-70b has the lowest at 28.34%.
• Iterative prompting with Solar feedback for three rounds reduces GPT-3.5-turbo overall CBS from 60.58% to 8.77% while Pass@attribute rises from 66.60% to 85.66%.
• CoT and positive role-play CoT prompts do not significantly reduce bias and CoT increases overall CBS to 72.65% for GPT-3.5-turbo.

Measured effects reported in the paper

Surprising findings

• Chain-of-thought prompting with fairness instructions increased bias for GPT-3.5-turbo rather than reducing it.
• Role-play prompting sometimes reinforced biases when sensitive attributes were embedded in the reasoning steps.

Limitations

• Dataset covers only seven demographic dimensions and 343 tasks in seven categories, limiting coverage of real-world scenarios.
• Pass@attribute is a lenient, attribute-based functional correctness metric rather than true functional correctness.
• Mitigation results are reported in detail only for GPT-3.5-turbo; other models' results are in an external artifact.

Experimental setup

Datasets: HumanEval

Benchmarks: Solar code bias dataset HumanEval

Metrics: Code Bias Score (CBS) Bias Leaning Score (BLS) Pass@attribute pass@1

Baselines: CoT prompting positive role-play + CoT prompting

Builds on: metamorphic testing textX HumanEval

Closest papers by meaning

• Prompting Fairness: Integrating Causality to Debias Large Language Models (ICLR 2025 · similarity 0.87)
• SocialStigmaQA: A Benchmark to Uncover Stigma Amplification in Generative Language Models (AAAI 2024 · similarity 0.87)
• Evaluating the Efficacy of Prompting Techniques for Debiasing Language Model Outputs (Student Abstract) (AAAI 2024 · similarity 0.86)
• FairMT-Bench: Benchmarking Fairness for Multi-turn Dialogue in Conversational LLMs (ICLR 2025 · similarity 0.85)
• StereoTales: A Multilingual Framework for Open-Ended Stereotype Discovery in LLMs (NeurIPS 2026 · similarity 0.85)
• MINT: Evaluating LLMs in Multi-turn Interaction with Tools and Language Feedback (ICLR 2024 · similarity 0.85)