Research map / Large language models / Language, safety and interpretability
Underspecification in Language Modeling Tasks: A Causality-Informed Study of Gendered Pronoun Resolution
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a causal account of how task underspecification induces latent selection bias in LLMs, and uses two black-box methods (gender-vs-time/location correlation measurement and a Task Specification Metric) to reveal such spurious correlations and detect underspecified gendered pronoun resolution tasks across 15 models.
Problem
Language modeling tasks are often underspecified at inference time, so models can learn spurious correlations (here gender vs. time/location) via sample selection bias that are not present in well-specified tasks; there is no simple black-box way to detect such underspecification.
Approach
A causal DAG with a selection mechanism shows that when features contain no causes of the label, selection bias on gender-neutral variables (time, location) induces gender-pronoun correlations; this is exploited by injecting time/location tokens and measuring shifts in pronoun probabilities, both to measure spurious correlations and to detect unspecified tasks with black-box access.
Key results
- Task Specification Metric on Winogender reaches about 84% balanced accuracy for RoBERTa-large and GPT-3.5 SFT (0.84 and 0.85).
- On Winogender Simplified, GPT-3.5 SFT, GPT-3.5 RLHF and GPT-4 Turbo Preview achieve balanced accuracy above 85% (0.96, 0.91, 0.86).
- Gender vs. time/location spurious correlations appear across models, with model size showing little effect while SFT and RLHF have the largest influence.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Apply the Task Specification Metric to Winogender | RoBERTa-large and GPT-3.5 SFT · Winogender | balanced accuracy: 0.84; 0.85 | Helped |
| Apply the Task Specification Metric to Winogender Simplified | GPT-3.5 SFT, GPT-3.5 RLHF, and GPT-4 Turbo Preview · Winogender Simplified | balanced accuracy: 0.96; 0.91; 0.86 | Helped |
| Increase model parameter count | LLMs evaluated with the MGC set · MGC gender vs. time and gender vs. location | spurious-correlation magnitude: 1,000× | No clear effect |
| Add post-training stages, particularly RLHF | LLMs evaluated with the MGC set · MGC gender vs. time and gender vs. location | spurious-correlation magnitude | Mixed |
| Move from GPT-3.5 SFT to later GPT-family models | GPT-family models · Winogender | balanced accuracy | Hurt |
| Use multiple and varied pre-training objectives | BART and UL2-family models · MGC gender vs. date and gender vs. location | linear-fit slope | Helped |
Surprising findings
- Model parameter count spanning about 1000x has relatively little influence on the magnitude of the specification-induced spurious correlations.
- Detection accuracy on GPT-family models declines after GPT-3.5 SFT on Winogender, possibly due to training exposure to the Winogender benchmark.
Limitations
- Detection threshold (0.5) and prompts were unoptimized and chosen by spot-checking; detection is near chance for some models.
- Method requires top-5 softmax or logprob access, so it is not fully black-box for all APIs; many factors vary across models, so the role of training objectives is speculative.
- Assumes binary gender and gender-neutral time/location values are time-invariant in the real world; dataset is synthetic templates.
Experimental setup
Datasets: Winogender Masked Gender Challenge (MGC)
Benchmarks: Winogender Winogender Simplified Masked Gender Challenge (MGC)
Metrics: slope of gender vs. time/location correlation Task Specification Metric true positive rate true negative rate balanced accuracy
Baselines: Winogender
Builds on: Winogender causal mediation analysis M-bias / selection bias theory
Closest papers by meaning
- ImpScore: A Learnable Metric For Quantifying The Implicitness Level of Sentences (ICLR 2025 · similarity 0.86)
- Exploring Activation Patterns of Parameters in Language Models (AAAI 2025 · similarity 0.85)
- Can Large Language Models Infer Causation from Correlation? (ICLR 2024 · similarity 0.85)
- Biases Mitigation and Expressiveness Preservation in Language Models: A Comprehensive Pipeline (Student Abstract) (AAAI 2024 · similarity 0.85)
- Latent Principle Discovery for Language Model Self-Improvement (NeurIPS 2025 · similarity 0.85)
- Evaluating the Zero-shot Robustness of Instruction-tuned Language Models (ICLR 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.