Research map / Large language models / AI and society

Position: Rethinking LLM Bias Probing Using Lessons from the Social Sciences

ICML 2025 · Large language models › AI and society · approach: Data commons and human baselines

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

A position paper that surveys LLM gender bias probes and proposes EcoLevels, a framework using ecological validity and abstraction level (associations, task-dependent decisions, naturalistic output) grounded in social science to select probes, reconcile conflicting results, and generate testable hypotheses.

Problem

Researchers face a proliferation of LLM social bias probes without principled criteria for choosing probes, a way to reconcile conflicting results across probes, or formal frameworks for predicting generalization to real user behavior.

Approach

Borrow ecological validity, boundary conditions, and the correspondence principle from social science to classify output-level gender bias probes by abstraction level (associations, task-dependent decisions, naturalistic output) and use probe-task alignment to select probes and interpret conflicts.

Key results

• Surveys roughly two dozen gender bias probes and classifies them into EcoLevels; for gender-occupation RQ1 narrows candidates from 24 to 12 association-level probes, and for RQ2 narrows to 7 naturalistic-output probes.
• Argues that association-level probes (LLM IB, WinoBias) show strong gender-occupation bias while naturalistic probes (LLM BTA, Resume Classification) show no significant bias, illustrating conflicting conclusions across probes.
• Generates four testable hypotheses, e.g., association-level probes should correlate more with real workforce gender distributions and naturalistic probes should show the most model-to-model variability due to RLHF.

Surprising findings

• Probes targeting gender bias yield conflicting conclusions: association-level probes show strong bias while naturalistic-output probes show none.
• Implicit (WEAT-style) associations correlate with language representations but explicit attitudes do not, questioning whether the implicit/explicit distinction applies to LLMs.

Limitations

• Levels form a continuum, so borderline cases (e.g., sentence completion with implied task) are subjective.
• Scope restricted to gender bias probes adaptable to prompt-to-output contexts.
• Framework is conceptual and its predictions are proposed as hypotheses rather than empirically tested.

Experimental setup

Baselines: WEAT CrowS-Pairs BBQ BiasInBios WinoBias WinoGender

Builds on: Word Embedding Association Test Implicit Association Test Winobias Winogender

Closest papers by meaning

• Rethinking Psychometric Evaluation of LLMs: When and Why Self-Reports Predict Behavior (NeurIPS 2026 · similarity 0.86)
• Cognitive Bias and Reassignment: Who Can Contribute High Quality LLM Data (AAAI 2025 · similarity 0.86)
• StereoTales: A Multilingual Framework for Open-Ended Stereotype Discovery in LLMs (NeurIPS 2026 · similarity 0.85)
• Prompting Fairness: Integrating Causality to Debias Large Language Models (ICLR 2025 · similarity 0.85)
• Position: LLM Social Simulations Are a Promising Research Method (ICML 2025 · similarity 0.84)
• Decision-Making Behavior Evaluation Framework for LLMs under Uncertain Context (NeurIPS 2024 · similarity 0.84)