Research map / Large language models / Language, safety and interpretability

Certifying Counterfactual Bias in LLMs

ICLR 2025 · Large language models › Language, safety and interpretability · approach: Bias measurement and mitigation · 3 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes LLMCert-B, a black-box probabilistic certification framework that gives Clopper-Pearson high-confidence bounds on the probability of unbiased (counterfactual) LLM responses over distributions of prefixed prompt sets.

Problem

Benchmarking and adversarial attacks cannot scale to or provide guarantees about counterfactual bias in LLM responses across demographic groups for large prompt distributions.

Approach

Specify counterfactual bias as the probability of unbiased responses over distributions of counterfactual prompt sets created by prepending prefixes (random tokens, mixtures of manual jailbreaks, embedding-space perturbations of jailbreaks), and certify it with Clopper-Pearson confidence intervals from iid samples, requiring only black-box query access.

Key results

• Mixture-of-jailbreak prefixes drive Mistral-7B's unbiased-response bound on BOLD to (0.22, 0.42) and soft prefixes to (0.30, 0.52), versus ~0.92-1.0 for random prefixes.
• GPT-3.5 gets a mixture-of-jailbreak bound of (0.44, 0.67) on BOLD and (0.66, 0.88) on DT, while GPT-4 gets (0.80, 0.96) on BOLD.
• The regard-based bias detector matched human judgments 76% of the time on 490 completion pairs, with 93% precision and 50% recall.

Measured effects reported in the paper

Surprising findings

• Certified bounds show GPT-4 is far less safe on BOLD mixture-of-jailbreak specifications (0.80-0.96) than the baseline without prefixes suggests, while the baseline reports near-100% unbiased responses.
• Model size does not show a consistent trend with fairness across specifications, suggesting alignment quality matters more than scale.

Limitations

• Bias detectors have imperfect accuracy (76% agreement with humans, 50% recall on BOLD), so bounds may be optimistic.
• Soft-prefix specifications require embedding access and cannot be applied to closed-source models; closed-source models were certified only for random and mixture prefixes.
• Certificates are for average-case behavior over prefix distributions, not worst-case adversarial prefixes, and use only binary gender/race groups in main experiments.

Experimental setup

Datasets: BOLD Decoding Trust stereotypes

Benchmarks: BOLD (250 profession prompts) Decoding Trust stereotypes (48 statements)

Metrics: Clopper-Pearson lower and upper bounds on probability of unb percentage unbiased without prefix percentage unbiased with main jailbreak

Baselines: Benchmarking without prefix Benchmarking with main jailbreak

Builds on: Clopper-Pearson confidence intervals Counterfactual Fairness Regard metric Decoding Trust stereotype benchmark

Closest papers by meaning

• Mission Impossible: A Statistical Perspective on Jailbreaking LLMs (NeurIPS 2024 · similarity 0.87)
• B-score: Detecting biases in large language models using response history (ICML 2025 · similarity 0.87)
• Investigating and Mitigating Undesirable Biases in Large Language Models (AAAI 2025 · similarity 0.86)
• Justice or Prejudice? Quantifying Biases in LLM-as-a-Judge (ICLR 2025 · similarity 0.86)
• Statistical Hypothesis Testing for Auditing Robustness in Language Models (ICML 2025 · similarity 0.86)
• All Should Be Equal in the Eyes of LMs: Counterfactually Aware Fair Text Generation (AAAI 2024 · similarity 0.86)